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FOREWORD  FROM  THE  CONFERENCE  CHAIRMAN 


Proceedings  of  the  2nd  Interservice/Industry  Training  Equipment  Confer-  / 
|"ehce  jjeld  in  Salt  Lake  City,  Utah,  November  18  through  November  20;  198# 

As  you  read  or  seek  reference  from  jt  after  the  conference^  I  want  to  be  presumptuous 
that  at  least  two  thoughts  will  be  rekindled. 


One  that  the  conference  program  was  current  and  exciting  and  another  more  en¬ 
compassing,  that  the  dedication,  long  contained  in  the  history  of  this  conference,  to 
making  it  THE  Training  Equipment  Conference  was  shown  to  be  continuing,  even  in- 
creasing.  0i?  fa  .  (gTj  5  ± Jj 

Ready,  willing  or  not,  we  are  launched  headlong* into  the  decade  of  the  80’s;  all  the 
challenges  projected  by  the  prophets  of  the  70’s  abound  and  then  some!  Weapon  sys¬ 
tems  are  more  complex;  the  need  for  well  designed  training  systems  and  associated 
training  equipment  more  vital.  It  is,  therefore,  incumbent  upon  all  of  us  in  the  training 
equipment  community  to  see  that  we  are  geared  up  to  meet  this  need.  This  training 
equipment  conference  with  the  information  we  share  with  each  other  represents  a  key 
step  toward  meeting  that  objective. 


There  is  a  caution  we  in  our  business  must  observe  as  we  prepare  to  meet  the  tasks  of 
the  80’s  and  beyond.  Our  colleagues,  the  operational  equipment  designers,  talk  of  ac¬ 
complishments  and  predict  progress  relating  to  systems  with  tremendous  degrees  of 
automation  and  high  orders  of  reliability.  We  must  guard  against  the  lulling  effect 
associated  with  these  words  that  can  cause  a  thought  posture  that  the  need  and  im¬ 
portance  of  training  will  diminish—even  disappear.  NOT  SO!  Man  is  and  will  be  firmly 
in  the  loop.  These  new  developments  do  help  him;  properly  designed  they  do  expand 
his  capabilities,  sometimes  lighten  his  task  load.  The  requirement  for  training  the 
operator  to  handle  the  systems  available  so  well  that  those  actions  he  must  take  are 
almost  involuntary  continues  to  exist. 

Training  equipment  design  will  also  continue  to  be  required  to  consider  the  situations 
that  will  exist  when  the  operator  must  be  trained  to  complete  his  mission  (and  here 
our  associates  who  design  operational  systems  will  grind  their  teeth)  when  the  opera¬ 
tional  system  is  in  a  degraded  performance  mode  or  even  a  partial  failure  mode. 

This  sums  up  to  all  of  us  in  the  business  taking  action  to  see  that  there  is  a  continuing 
and  expanding  consciousness  of  the  value  and  need  for  training  equipment.  This  con¬ 
ference  and  these  proceedings  represent  a  significant  portion  of  that  action. 


R.  W.  Layne 

NSIA  Conference  Chairman 
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ABSTRACT 


The  rapid  advancement  In  capabilities  of  computer  image  generation  (CIG)  visual 
systems  has  resulted  in  Increased  application  of  such  systems  from  take-off  and  landing 
training  to  full-mission  simulation.  These  full-mission  applications  dictate  a  need 
for  the  creation  of  high-fidelity  data  bases  covering  large  geographical  areas  on  the 
order  of  multiple  thousands  of  square  miles.  As  a  consequence,  the  manual  techniques 
employed  in  the  past  to  create  small,  airfield  vicinity  data  bases  are  not  practical 
for  generation  of  very  large  data  bases.  Clearly,  some  type  of  automated  data  base 
generation  technique  is  required.  Current  trends  are  aimed  at  utilizing  the  Defense 
Mapping  Agency  (DMA)  digital  data  base  as  source  data  and  applying  to  it  a  computer 
transformation  in  order  to  arrive  at  a  real-time  CIG  data  base.  However,  current 
limitations  of  the  DMA  data,  CIG  system  constraints,  and  training  utility  of  the  end 
product  limit  the  amount  of  automation  possible  in  the  data  base  generation  process. 
This  paper  discusses  the  evolution  of  the  CIG  data  base  generation  process  from 
totally  manual  methods  to  current  trends  toward  full  automation.  Practical  limits 
of  automation  and  potential  future  developments  are  examined. 


INTRODUCTION 

Since  late  1973  when  the  cost  of  aviation 
fuel  began  to  rise  and  availability  became  less 
certain,  the  Air  Force  has  endeavored  to  reduce 
the  number  of  training  hours  in  the  actual  air¬ 
craft.  Accordingly,  the  full-mission  considera¬ 
tions  of  simulator  training  have  become  more 
important.  Visual  simulation  has  become  a  major 
contributor  to  the  full-mission  training  capa¬ 
bility  in  simulators.  The  most  widely  used  method 
for  generation  of  the  visual  Imagery  for  Air  Force 
simulators  is  that  of  Computer  Image  Generation 
(CIG).  This  technology  employs  high-speed, 
special  purpose  computational  logic  to  generate 
perspective  imagery  using  a  mathematical  model  of 
the  real-world  as  input  data  (data  base).  The 
problems  encountered  in  determining  the  content 
of  the  data  base,  as  well  as  limitations  in  the 
generation  of  the  data  base  and  the  resulting 
impact  on  training  capability  are  the  subject  of 
this  paper. 

WHERE  HAVE  WE  COME  FROM? 

In  the  Beginning 

A  classical  measure  of  the  processing  capa¬ 
bility  of  the  CIG  system  is  the  edge  capacity. 

An  edge  is  defined  as  the  boundary  between  two 
separate  surfaces  in  the  displayed  imagery.  CIG 
systems  acquired  during  the  infancy  of  the  tech¬ 
nology  (roughly  1962-1972)  by  NASA  and  the  U.S. 
Navy  could  generate  on  the  order  of  240  to  512 
edges.  The  data  bases  used  by  these  systems  were 
thus  very  small  and  low  in  detail.  These  data 
bases  were  all  hand-modeled  at  relatively  low  cost 
in  manhours.  Their  small  size  and  simplicity  made 
testing  and  debugging  (correcting  anomalies  which 
cause  the  generation  of  incorrect  scenes  or  dis¬ 
tracting  effects)  a  relatively  simple  task.  How¬ 
ever  these  early  CIG  systems,  because  of  their 
limited  power,  were  used  for  training  only  basic 
flying  maneuvers  such  as  take-off  and  landing. 


In  the  late  sixties  and  early  seventies  the 
dramatic  progress  in  the  electronic  microcircuit 
industry  was  beginning  to  reflect  on  CIG  tech¬ 
nology.  Image  generators  capable  of  processing 
1000  to  2000  edges  became  available.  Along  with 
this  growth  in  image  generator  capacity  came  the 
attendant  growth  in  edge  density  and  in  the  geo¬ 
graphical  area  of  coverage  of  the  data  base.  The 
Air  Force's  Advanced  Simulator  for  Pilot  Training 
(ASPT)  was  perhaps  the  premier  system  developed 
during  this  time.  The  ASPT  visual  system  had  the 
capability  to  process  up  to  2500  edges  from  an 
originally  delivered  data  base  containing  approx¬ 
imately  100,000  edges.  The  initial  ASPT  data 
bases  covered  an  area  of  500  square  nautical  miles 
and  yet  were  still  generated  by  hand  using  engin¬ 
eering  drawings,  blueprints,  photos,  maps  and 
personal  observations.  Those  features  or  objects 
which  were  deemed  significant  as  cues  for  train¬ 
ing  were  manually  extracted  one-by-one  from  the 
source  data.  The  time  required  to  intelligently 
fill  a  data  base  of  this  size  with  sufficient 
edges  and  lights  to  fully  exploit  the  capability 
of  this  more  powerful  image  generator  was  becoming 
significant.  The  ratio  of  data  base  cost  to  the 
total  cost  of  the  visual  system  was  starting  to 
rise. 

The  Air  Force's  Advanced  Electro-Optical 
Sensor  Simulation  system  (AEOSS)  was  developed  in 
the  mid-seventies  as  a  tool  to  investigate  not 
only  E-0  sensor  simulation  using  CIG,  but  new 
image  generation  techniques  as  well.  Although  the 
AEOSS  was  a  non-real  time  research  oriented  device, 
it  had  the  capability  to  replay  relatively  com¬ 
plex  Imagery  (containing  over  seven  thousand 
edges)  in  real-time  for  evaluation.  Perhaps  the 
real  significance  of  this  system,  however,  was 
that  it  marked  the  first  application  of  a  fully 
automated  technique  for  CIG  data  base  generation. 
The  terrain  definition  for  the  AEOSS  data  base 
was  derived  by  fitting  surfaces  to  the  terrain 
elevation  data  provided  by  the  Defense  Mapping 
Agency  (DMA)  Digital  Land  Mass  Simulation  (DLMS) 
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data  base.  DMA  also  provides  a  cultural  data  file 
to  describe  the  topography  of  the  terrain,  but  for 
AEOSS  it  was  used  mainly  to  define  the  locations 
of  large  water  features  (lakes,  etc.)  on  the  ter¬ 
rain.  Some  of  the  man-made  cultural  features  were 
also  transformed  automatically,  but  this  was  done 
using  a  relatively  gross  approximation  of  the  fea¬ 
tures  with  little  detail.  All  of  the  detailed  cul¬ 
ture  specifically  required  for  the  AEOSS  system 
was  modeled  and  inserted  into  the  data  base  by 
hand.  This  becomes  significant  when  one  considers 
that  the  geographic  area  of  coverage  has  been 
extended  to  2500  square  nautical  miles. 

More  Systems 

Although  CIG  technology  was  struggling  to 
its  feet  in  the  early  seventies,  the  impending 
fuel  shortage  was  about  to  force  CIG  to  take  a 
giant  leap  forward.  The  1973  Mid-East  oil  embargo 
had  a  significant  effect  on  the  price  and  avail¬ 
ability  of  jet  aviation  fuel.  The  immediate 
reality  for  the  Air  Force  and  the  rest  of  DOD  was 
a  gradual  yet  sizeable  cutback  in  the  amount  of 
flying  hours  normally  earmarked  for  training.  The 
answer  -  use  simulators  to  replace  the  lost  train¬ 
ing  hours.  The  Air  Force  then  embarked  on  a 
series  of  new  simulator  acquisition  programs. 

More  Performance 

Since  in  many  cases  the  new  simulators  were 
to  replace  and  no  longer  simply  augment  training 
in  the  actual  aircraft,  procurement  specifications 
were  written  for  devices  having  significantly 
higher  performance  capabilities  than  existing 
simulators  of  that  time.  Inherent  in  these  spec¬ 
ifications  was  a  "full  mission"  training  capa¬ 
bility;  the  ability  to  rehearse  a  total  mission 
from  take-off  to  landing  -  all  in  the  simulator. 
Visual  simulation  was  a  key  to  this  capability. 
Take-off  and  landing  and  general  cockpit  pro¬ 
cedures  proficiency  were  no  longer  the  only  simu¬ 
lator  training  tasks  to  be  dealt  with.  Low  level 
navigation,  aerial  refueling,  visual  weapons 
delivery  and  aerial  combat  training  via  simulators 
were  now  firm  requirements.  Because  of  its 
inherent  flexibility  and  rapidly  growing  capa¬ 
bility,  CIG  was  considered  the  visual  simulation 
technology  most  capable  of  meeting  these  new 
requirements . 

Along  with  the  more  powerful  CIG  systems 
capable  of  supporting  full-mission  simulation, 
requirements  for  data  bases  covering  large  geo¬ 
graphic  areas  of  the  world  began  to  evolve. 

Perhaps  the  most  challenging  data  base  require¬ 
ments  for  a  CIG  visual  system  are  tied  . to  the 
Air  Force's  C-130  visual  system  procurement 
program. 

WHERE  ARE  WE  TODAY? 

Larger  Data  Bases 

The  C-130  visual  system  pilot  production 
unit  is  being  procured  with  three  separate  data 
bases.  One  of  these  data  bases  will  be  a  rela¬ 
tively  small  (2,500  sq  mile)  area  surrounding 
Barksdale  Air  Force  Base.  The  other  two  data 
bases  will  each  cover  40,000  square  nautical 
miles  in  the  vicinities  of  Little  Rock  (Arkansas) 


and  Kirtland  (New  Mexico)  Air  Force  Bases.  The 
latter  two  data  bases  will  contain  the  respective 
air  bases  as  well  as  airdrop  zones  to  allow  for 
training  in  the  following  areas: 

1 .  Ground  operation 

2.  Takeoff,  approach  and  landing 

3.  Formation  flight 

4.  Low  level  navigation 

5.  Assault  landing 

It  is  the  low  level  navigation  requirement  along 
with  the  large  area  of  coverage  which  necessitates 
new  data  base  generation  techniques. 

Since  the  C-130  flight  simulator's  radar 
landmass  system  was  already  using  the  DMA  DLMS 
data  base  as  source  data,  the  decision  was  made 
to  utilize  the  DLMS  data  base  as  one  input  for 
the  visual  system  data  base  generation  process 
as  well.  The  one  obvious  benefit  to  be  derived 
from  using  the  common  data  source  was  correlation 
between  the  radar  and  visual  subsystems,  ie. 
terrain  prominences  and  cultural  features  will  be 
portrayed  in  the  same  geographic  locations  in 
both  systems.  The  second  and  perhaps  more  signifi¬ 
cant  benefit  to  be  drawn  from  the  use  of  the  DLMS 
data  base  is  its  digital  formal.  No  manual  digi¬ 
tization  would  be  required  in  order  to  insert 
cultural  features  found  in  the  DMS  data  base  into 
the  CIG  visual  system  data  base,  although  the 
transformation  program  had  to  be  expanded  to 
provide  this  capability. 

Thus,  the  complexity  of  a  transformation 
program  which  formerly  emphasized  terrain  trans¬ 
formation  has  been  increased  significantly  with 
the  addition  of  a  more  faithful  culture  transfor¬ 
mation  and  the  process  of  merging  the  resulting 
terrain  and  culture  models.  Even  with  this  ex¬ 
panded  capability,  an  enormous  expenditure  in  man¬ 
hours  is  required  to  customize  the  data  base  for 
training  by  manually  adding  training  significant 
features  not  found  in  the  DMA  DLMS  culture  data. 

Source  Data  Limitations 

Truly,  the  final  data  base  product  can  only 
be  as  good  as  the  source  data  used  to  generate  it. 
In  fact  the  source  data  is  the  first  problem 
encountered  in  the  data  base  generation  process. 
DMA's  DLMS  data  base  is  the  most  comprehensive 
source  of  data  currently  available  for  use  as 
source  data  for  CIG  data  bases.  The  DLMS  data 
base  is  a  good  source  of  data  because  it  includes 
both  real-world  terrain  elevation  and  cultural 
data,  and  will  eventually  be  expanded  to  cover 
literally  millions  of  square  miles,  including  the 
continental  U.S.  as  well  as  foreign  countries. 

Although  the  large  area  of  coverage  of  the 
DMA  DLMS  data  base  makes  it  attractive  for  use  as 
source  data  for  visual  data  bases,  one  significant 
drawback  does  exist  in  the  fact  that,  from  the 
outset,  the  DLMS  data  base  has  been  created  to 
primarily  support  radar  simulation.  Hence  only 
larger  "radar  significant"  features  are  included 
In  the  culture  file.  Accordingly  the  DLMS  data 
base  does  not  Include  many  types  of  features  which 
are  very  significant  from  a  visual  navigation 
standpoint.  This  Includes  such  features  as  roads, 
overpasses,  railroads,  and  small  streams. 
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Another  limitation  In  the  DLMS  data  base  is 
a  lack  of  any  coloration  or  surface  characteris¬ 
tic  (texture)  information.  Also  the  size  criteria 
which  DMA  uses  to  capture  features  in  the  DLMS 
production  process  limits  the  resolution  of  the 
displayed  visual  data  bases  to  much  less  than  that 
which  is  resolvable  in  current  visual  displays. 
Consequently,  any  area  in  the  data  base  which 
requires  high  detail  must  be  enhanced  via  a  time 
consuming  manual  process  using  other  sources  of 
data . 

Thus,  there  exists  no  single  totally  compre¬ 
hensive  source  of  data  which  provides  the  detail 
and  accuracy  required  to  model  even  take-off  and 
landing  type  data  bases,  much  less  data  bases  to 
support  the  low-level  navigation  tasks  required 
of  the  C-130  system.  While  the  obvious  solution 
is  to  selectively  utilize  inputs  from  a  combina¬ 
tion  of  several  different  data  sources,  it  is 
precisely  this  solution  which  introduces  some  of 
the  major  problems  currently  being  encountered 
in  the  generation  of  large  data  bases. 

Registration 

Since  certain  areas  of  CIG  data  bases  often 
contain  navigationally  significant  features  which 
are  not  completely  defined  in  the  DMA  DLMS  data, 
these  features  must  be  obtained  from  a  different 
source.  There  are  alternate  data  sources  such 
as  Joint  Operation  Graphics  (JOG)  charts.  Tactical 
Pilotage  Charts  (TPC),  photos,  etc.  However, 
when  additional  features  are  extracted  from  any 
of  these  alternative  data  sources  and  inserted 
into  the  previously  transformed  data,  registra¬ 
tion  with  other  features  in  the  data  base  can 
become  an  almost  insurmountable  problem  unless 
proper  care  is  taken  at  the  correct  stage  of  the 
data  base  generation  process.  A  case  in  point  is 
the  C-130  visual  system  data  bases  in  which 
bridges  are  extracted  from  the  DMA  data,  and 
streams  and  roads  are  placed  in  the  on-line  data 
base  using  JOG  charts  as  source  data.  When 
extracting  features  from  two  different  sources, 
each  with  its  own  accuracy  tolerance,  the  result 
is  predictable.  For  instance,  a  road  may  run  up 
to  a  stream  and  miss  the  bridge  by  several  hundred 
yards,  if  in  fact  the  bridge  traverses  the  stream 
at  all.  Several  other  such  misregistration  situ¬ 
ations  are  possible,  such  as: 

1.  Dams  to  lakes 

2.  Streams  to  lakes 

3.  Roads  to  water  features 

These  irregularities  are  unacceptable  for  naviga¬ 
tional  training  and  potentially  require  signifi¬ 
cant  numbers  of  interactive  manhours  to  insure 
proper  registration.  Manual  interaction  is  a 
leading  contributor  to  data  base  production  costs, 
but  at  present  there  is  no  clear  alternative  to 
resolve  these  types  of  problems. 

Real-Time  Constraints 

On  the  other  end  of  the  data  base  generation 
issue  is  the  capability  of  the  image  generator  to 
handle  the  data  base  content.  Data  bases  must  be 
generated  with  consideration  of  the  target  image 
generator's  processing  capacity.  Failure  to  do  so 
will  more  than  likely  result  in  system  overload 
which  is  characterized  by  unacceptable  streaking 


or  breaking  up  of  the  video  image.  Thus,  the 
goal  in  the  development  of  any  particular  CIG 
system  is  to  create  a  uniformly  dense  data  base 
that  will,  in  general,  be  tailored  to  load  the 
image  generator  to  an  optimum  percentage  of  its 
capacity  without  overloading  it.  This  is  not 
typical  of  past  approaches  where  the  image  genera¬ 
tor  has  been  sized  to  handle  the  most  dense, 
worst  case  areas  of  the  data  base  which  usually 
constituted  a  relatively  small  percentage  of  the 
entire  geographical  area  of  coverage.  This 
approach  guaranteed  waste  of  a  large  percentage 
of  image  generator  capacity  whenever  sparsely 
populated  areas  of  the  data  base  were  traversed. 

Several  performance  parameters  of  CIG  image 
generators  need  to  be  considered  during  the  data 
base  generation  process.  The  most  important  of 
these  is  the  number  of  faces  or  edges  which  the 
system  can  handle.  This,  in  conjunction  with  the 
field-of-view  of  the  visual  system,  leads  to  an 
understanding  of  the  edge  or  face  density  required 
in  the  data  base  to  properly  load  the  image  gener¬ 
ator.  The  number  of  edge  crossings  per  raster 
line  that  the  image  generator  can  handle  is  also 
a  parameter  that  influences  allowable  edge  density 
in  the  data  base.  Finally,  there  is  the  consid¬ 
eration  of  mass  storage  for  the  data  base.  While 
not  a  problem  at  present,  future  systems  with 
larger  capacity  and  higher  fidelity  image  genera¬ 
tors  and  more  complex  data  bases  will  impose  new 
and  more  challenging  requirements  for  data  base 
storage  and  real-time  retrieval.  The  only  real 
concern  is  that  the  stored  data  base  is  adequate 
to  optimally  load  the  image  generator  and  is 
capable  of  being  accessed  rapidly  enough  to 
allow  real-time  generation  of  the  requi-ed  scenes. 

Other  Considerations 

Generation  of  acceptable  on-line  data  bases 
is  not  assured  by  simply  creating  mathematically 
correct  object  definitions  and  avoiding  conflict 
with  image  generator  capabilities.  Indeed  a  dis¬ 
played  data  base  may,  upon  initial  static  evalua¬ 
tion,  look  very  acceptable.  However,  when  the 
data  base  is  applied  to  a  real-time  training 
scenario,  the  same  displayed  data  base  may  be 
judged  to  be  unuseable.  When  dealing  with  CIG 
systems,  the  relative  acceptability  of  a  data 
base  and  the  concept  of  "realism"  are  not 
necessarily  related.  In  order  to  accomplish 
training,  the  data  base  must,  in  conjunction  with 
the  image  generator,  provide  adequate  and  "proper1' 
visual  cues  to  the  aircrew.  The  proper  cues  vary 
for  each  different  task.  For  take-off  and  land¬ 
ing,  the  relationship  of  the  runway  to  the 
surrounding  terrain  and  culture  and  the  texture 
or  character  of  the  terrain  surface  may  be  most 
important.  For  low  level  navigation  however,  the 
occlusion  of  cultural  features  by  other  cultural 
features  and  the  terrain  profile  is  more  signifi¬ 
cant.  Also  a  low  level  navigation  data  base 
must  include  a  sufficient  number  of  cultural 
features  to  clutter  the  scene  in  order  to  train 
the  navigator  to  discriminate  between  similar 
features  in  establishing  checkpoints. 

Another  requirement  for  navigational  train¬ 
ing  is  consistency  which  can  be  exemplified  by 
the  ability  of  a  student  navigator  to  look  at 
his  chart  and  base  the  determination  of  a  check¬ 
point  on  a  feature  on  the  chart,  with  full 
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confidence  that  the  feature  in  question  will 
appear  in  the  CIS  data  base  for  every  occurrence 
on  the  chart.  Dealing  with  this  problem  properly 
entails  numerous  hours  devoted  to  manual  pro¬ 
cesses.  The  pain  involved  can  be  eased  by 
intelligently  determining  what  features  are  nav- 
igationally  significant  and  including  all  such 
features  at  the  cost  of  eliminating  other  fea¬ 
tures  whose  presence  makes  the  data  base  look 
more  realistic,  but  adds  no  training  value.  Of 
course  the  determination  of  what  is  navigation- 
ally  significant  needs  to  be  determined  sepa¬ 
rately,  to  so roe  extent,  for  each  different  data 
base  area.  This  is  illustrated  on  the  C-130  by 
the  fact  that  the  Little  Rock  area  is  tree  cov¬ 
ered  with  numerous  lakes  and  a  moderate  amount 
of  man-made  cultural  detail.  The  Kirtland  area, 
on  the  other  hand,  contains  very  sparse  cultural 
content  (approximately  8%  of  the  number  of 
features  found  in  the  DMA  data  base  for  the 
Little  Rock  area  for  roughly  the  same  geograph¬ 
ical  area).  In  the  Little  Rock  area,  the  lakes 
and  trees  are  navigationally  significant,  while 
such  features  as  farms  and  ranches  are  useful 
for  their  clutter  value  but  are  of  secondary 
importance  for  overall  navigation.  In  the 
Kirtland  area,  however,  isolated  ranches  and 
windmills  comprise  a  large  percentage  of  the 
cultural  detail  and  are  primary  navigational 
cues . 

Coloration  of  features  is  a  significant 
navigational  cue  and  therefore  is  also  an  import¬ 
ant  consideration  in  data  base  generation. 

Subtle  color  differences  between  features  cannot 
always  be  successfully  inferred  from  source  data 
but  instead  must  sometimes  be  gleaned  from  per¬ 
sonal  observation  or  color  photography. 

One  last  consideration  in  the  generation 
of  large  area  data  bases  is  the  concept  of  homo¬ 
geneity.  The  displayed  data  base  must  possess 
a  distribution  of  features  (detail)  which  is 
analogous  to  the  real  world.  Concentration  of 
features  in  an  area  of  the  on-line  data  base 
where  the  same  relative  concentration  does  not 
exist  in  the  real  world  can  lead  to  artificial 
cueing.  Corridorizing  of  a  data  base  is  the 
practice  of  concentrating  data  base  detail  along 
a  pre-determined  flight  path.  While  corridoriz¬ 
ing  can  help  to  reduce  the  number  of  manhours 
required  to  build  a  data  base,  it  does  tend  to 
limit  the  flexibility  of  use  of  the  data  base 
and  can  also  result  in  artificial  cueing;  if 
most  of  the  detail  is  clustered  around  one  pre¬ 
determined  path  through  a  data  base,  other 
missions  with  other  routes  of  flight  cannot  be 
readily  accomplished.  Also  if,  while  flying  a 
mission  along  the  preselected  corridor. the  pilot 
begins  to  inadvertently  diverge  from  the  pre¬ 
scribed  path,  the  crew  will  immediately  be 
alerted  to  his  error  by  the  sudden  reduction  of 
scene  detail  away  from  the  corridor. 

In  many  regards  data  base  creation  is  as 
much  an  art  as  it  is  a  science  and  the  above 
comments  are  intended  to  illustrate  that  a  rigid, 
cookbook  approach  to  data  base  generation  is  not 
usually  sufficient.  Even  if  all  the  mechanical 
registration  type  problems  are  eventually  solved, 
it  appears  a  certainty  that  some  amount  of 
modeler  Interaction  during  the  data  base  produc¬ 
tion  process  will  always  be  required.  The  object- 
tive,  then,  must  be  to  minimize  modeler  inter¬ 


action  by  maximizing  the  amount  of  automation  used 
in  the  basic  feature  extraction  process  and  leave 
any  manual  interaction  to  debugging  and  fine  tun¬ 
ing  enhancements. 

WHERE  ARE  WE  HEADED? 

It  is  clear  that  there  are  many  unsolved 
problems  in  the  visual  data  base  generation 
process.  Furthermore,  the  current  problems  are 
only  likely  to  worsen  as  technology  advances  and 
image  generator  capacity  increases  along  with  the 
demands  on  the  data  base.  Thus,  data  base  pro¬ 
duction  costs  can  only  be  expected  to  keep  rising. 
What  can  be  done  about  it?  The  Air  Force  is 
currently  pursuing  several  efforts  in  order  to 
deal  with  these  problems. 

Data  base  commonality  is  one  potential  solu¬ 
tion  to  the  problem.  In  past  systems,  radar  and 
visual  data  bases  as  well  as  any  sensor  data  bases 
have  all  been  developed  independently  of  each 
other.  The  hand-modeled  areas  and  processes  that 
require  manual  interaction  are  and  will  continue 
to  be  fairly  expensive  parts  of  the  process.  It 
is  possible  that  generating  these  areas  once  and 
including  appropriate  features/descriptors  for 
visual,  sensor,  and  radar  all  in  the  same  data 
base  could  ease  the  data  base  generation  manpower 
requirements.  The  tradeoff  potentially  involves 
more  storage  for  the  on-line  data  base  if  para¬ 
meters  which  are  not  being  used  for  a  certain 
application  are  built  in.  However,  a  secondary 
translation  program  could  be  applied  to  extract 
a  specialized  data  base  from  the  universal  common 
data  base  and  thereby  eliminate  the  overhead. 

Data  base  portability  is  another  idea  under 
consideration  by  the  Air  Force  for  use  in  future 
systems.  This  concept  would  require  that  all 
future  data  bases  be  generated  to  the  highest 
detail  or  fidelity  possible  and  described  in  a 
high-order  type  data  base  language.  This  data 
base  could  then  be  compiled  (or  filtered)  for 
other  simulators  or  for  other  types  of  missions 
in  the  same  simulator  by  specialized  software 
which  uses  the  high  level  description  as  input 
and  develops  a  tailored  data  base  for  a  particu¬ 
lar  application  on  a  target  visual  system.  Each 
feature  in  the  high-level  data  base  would  contain 
appropriate  importance  descriptors  for  filter 
survival  based  on  the  type  of  mission  to  be 
trained  and  the  capacity  of  the  intended  image 
generator.  This  technique,  combined  with  a  dis¬ 
criminator  capability  in  the  image  generator  to 
monitor  image  generator  loading  and  select  fea¬ 
tures  from  the  on-line  data  base  in  priority  (or 
importance)  order,  should  save  on  data  base  gen¬ 
eration  expense  and  at  the  same  time  provide  for 
better  image  generator  loading  and  overload  pre¬ 
vention  in  CIG  visual  systems. 

There  is  also  the  consideration  of  future 
alternate  data  sources.  Although  at  present  the 
DLMS  data  base  is  the  best  source  of  Information 
available  for  CIG  data  bases,  there  is  the  possi¬ 
bility  that  new  and  better  sources  of  data  may 
be  developed  in  the  future.  One  must  keep  in 
mind,  however,  that  it  will  probably  take  require¬ 
ments  by  potential  users  other  than  the  simulator 
community  to  make  the  development  of  such  a 
"customized  for  training"  data  base  worthwhile, 
considering  the  great  expense  Involved  in  data 
base  construction. 
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One  future  possible  source  is  the  DMA  level  imately  ten  years. 

V  (for  visual)  data  base  currently  being  planned. 

Level  V  will  involve  the  addition  of  features  and  In  summary,  the  technology  of  computer  image 

descriptive  parameters  to  the  DLMS  data  base  for  generation  and  attendant  advances  in  data  base 
the  purpose  of  better  defining  features  and  sur-  generation  for  CIG  has  seen  tremendous  advances, 

face  characteristics  which  are  navigational ly  or  Many  more  advances  are  currently  being  envis- 

perceptually  significant  for  visual  system  use.  ioned,  planned  or  Implemented.  But  in  the  final 

Level  X  is  a  planned  extension  of  Level  V  which  analysis,  it  is  probable  that  some  degree  of 

will  provide  relatively  small  areas  of  high  manual  enhancement  or  editing  will  always  be 

detail  within  Level  V.  However  neither  Level  V  required  nc  matter  what  the  capacity  of  the 

nor  X  is  expected  to  be  implemented  (except  target  image  generator  or  the  intelligence  of 

possibly  for  demonstration  purposes)  for  approx-  the  transformation  program  is. 
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ABSTRACT 


This  visual  data  base  transformation  program  automatically  transforms  the  Defense  Mapping 
Agency  Digital  Data  Base  (DMA  DDB)  into  a  data  base  for  real-time  simulation  of  the  B-52 
electro-optical  viewing  system.  The  user  reviews  the  source  data  and  specifies  edge  budgets, 
error  tolerances,  and  visual/infrared  models  for  DMA  planimetry  feature  classes.  The  DMA  DDB 
is  read  and  reblocked  into  standard  geographical  areas;  data  from  different  manuscripts, 
levels,  and  releases  are  merged  into  a  composite  source  file.  The  user  may  edit  the  composite 
source.  Then  planimetry  and  terrain  are  independently  modelled  into  visual/IR  representations 
and  progressively  simplified  (levelled)  to  meet  the  specified  edge  and  error  budgets  at 
several  levels  of  detail.  The  levelled  planimetry  and  terrain  are  integrated  into  a  combined 
scene  model  and  reformatted  for  real-time  use.  Utility  software  facilitates  production 
management  and  configuration  control.  Significant  advances  include  automatic  level  of  detail 
generation,  terrain  modelling,  planimetry/ terra  in  integration,  and  a  100-fold  speedup  in  data 
base  generation. 


INTRODUCTION 

As  flight  simulator  training  scenarios  be¬ 
come  more  complex  and  extensive,  so  must  the 
visual  data  bases  which  support  them,  straining 
the  capabilities  of  manual  modelling  techniques. 
The  LINK  Division  of  the  Singer  Company  has 
developed  for  the  B-52  Weapons  System  Trainer 
(WST)  a  computer  program  which  automatically 
transforms  the  DMA  planimetry  and  terrain  data 
bases  into  a  data  base  which  supports  real-time 
Simulation  of  visual  and  infrared  (IR)  sensor 
displays  (1,  2,  3).  This  paper  describes  the 
concepts  and  operation  of  the  transformation 
program,  illustrates  the  transformation  of  a 
typical  area  as  shown  in  Figure  la,  and 
discusses  the  advantages  and  problems  of 
automatic  transformation. 


The  B-52  WST  is  designed  to  provide 
full-mission,  full-crew  training,  including  day 
and  night  low-level  penetration  during  Emergency 
War  Order  and  training  missions.  Low-level 
penetration  includes  terrain  avoidance,  low-level 
navigation,  bomb  damage  assessment,  and  low-level 
gravity  weapon  delivery.  These  tasks  are 
supported  by  visual  (out-the-window)  and 
electro-optical  sensor  displays  within  a  mission 
corridor  of  up  to  130,000  square  miles.  (Other 
mission  phases  and  aircraft  systems  are  simulated, 
but  are  not  important  to  this  paper.) 

The  electro-optical  viewing  system  (EVS) 
consists  of  forward-looking  infrared  (FUR)  and 
steerable  low-light  level  television  (STV)  cameras 
in  two  pods  under  the  aircraft  nose,  and  control 
and  display  consoles  at  the  pilot,  copilot,  and 
radar  navigator  stations.  Both  sensors  can  be 
steered  in  elevation  and  azimuth  and  provide  wide 
and  narrow  fields  of  view.  The  FUR  is  sensitive 
to  both  reflection  and  thermal  emission;  diurnal 
variations  are  apparent  and  are  included  in  the 
simulation. 

The  simulated  visual,  STV,  and  IR  scenes  are 
generated  in  real-time  by  the  VISULINK  ®  Digital 
Image  Generator  (DIG).  The  DIG  extracts  from  a 
visual  data  base  the  parts  of  the  scene  relevant 
to  the  pilot  eyepoint  computed  by  the  flight 
simulation;  computes  a  correct  perspective  view; 
removes  distant  objects  occulted  by  near  objects; 
and  displays  the  resulting  scene.  The  same  data 
base  is  used  for  visual,  STV,  and  IR  simulation; 
separate  color,  intensity,  and  IR  codes  are 
provided,  but  the  geometrical  model  and  all  other 
components  are  the  same. 
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The  visual  and  E VS  simulations  are  correlated 
with  flight  simulation  and  other  aircraft  systems 
in  real-time  during  training;  a  stand-alone  mode 
allows  independent  operation  from  a  diagnostic 
control  panel . 

Dynamic  scene  levelling  simplifies  those 
parts  of  the  scene  far  from  the  eyepoint;  this 
concentrates  detail  in  the  foreground  and  makes 
the  most  efficient  use  of  DIG  capacity.  Several 
levels  of  detail  are  constructed  during  transfor¬ 
mation;  one  is  selected  and  displayed  during 
real-time  operation. 

Planimetry  data  is  provided  on  magnetic 
tape  by  the  Defense  Mapping  Agency /Aerospace 
Center  (DMAAC)  and  defines  planimetric  features 
such  as  lakes,  cities,  forests,  roads,  and 
buildings  in  digital  form  (4).  Level  I  data 
covers  large  areas  of  the  earth's  surface  at  low 
resolution;  Level  II  data  covers  important  small 
areas  at  high  resolution.  Each  tape  typically 
covers  a  rectangle  in  latitude/longitude  from 
1°  X  1°  to  1°  X  6°  and  is  divided  into  manuscripts 
ranging  from  7V  X  7V  to  30'  X  15 ‘ .  Each 
manuscript  describes  all  the  planimetric  features 
within  its  defined  area.  Figure  lb  shows  a  sample 
manuscript  corresponding  to  the  outlined  area  in 
Figure  la. 


Figure  lb 

Example  Planimetry  Manuscript 

A  feature  is  uniquely  identified  within  its 
manuscript  by  a  feature  analysis  code  (FAC  #). 

If  two  features  overlap,  the  one  with  the  higher 
FAC  0  masks  the  other.  The  predominant  height 
of  the  feature  gives  its  characteristic  height 
above  the  terrain  or  underlying  feature.  The 
feature  identification  code  (FID)  describes  the 
kind  of  feature:  quarry,  smokestack,  railroad, 
bridge,  residence,  navigation  aid,  runway,  lake, 
shoreline,  etc.  The  surface  material  code  (SMC) 
describes  the  composition;  metal ,  stone,  water, 
sand,  soil,  marsh,  snow/ice,  etc. 

The  feature  type  distinguishes  different 
feature  geometries:  a  point  feature  is  digitized 
as  a  set  of  isolated  points  supplemented  by 
orientation,  length,  width,  height,  FID,  and 
SMC.  A  lineal  feature  is  digitized  as  a  chain 
or  open  polygon  supplemented  by  width,  height, 
FID,  and  SMC.  An  areal  feature  is  digitized  as 
a  closed  counter-clockwise  polygon  supplemented 
by  height,  FID,  and  SMC. 


Terrain  data  is  also  provided  by  DMAAC  and 
defines  terrain  elevations  in  digital  form  (4). 
Each  tape  covers  from  1°  X  1°  to  1°  X  6°;  a 
manuscript  always  covers  1°  X  1°.  The  terrain 
elevations  are  sampled  at  a  rectangular  grid  in 
latitude  and  longitude.  The  grid  spacing  in 
latitude  is  3  arc-seconds  (about  300  ft);  the 
grid  spacing  in  longitude  varies  from  3  to  18 
arc-seconds  (about  300  ft)  depending  on 
latitude.  Unknown  elevations  are  designated  by  a 
special  code.  Figure  1c  shows  a  sample 
manuscript.  (The  grid  spacing  is  distorted  for 
clarity.) 
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Figure  1c 

Example  Terrain  Manuscript 


The  ranainder  of  this  paper  describes  the 
program  which  transforms  the  DMA  DDB  into  a 
visual  data  base  for  real-time  simulation.  It 
outlines  the  organization  of  the  program  and  of 
its  intermediate  and  output  data  bases; 
describes  the  processing  steps  performed; 
discusses  the  significant  advances  to  the  state 
of  the  art;  and  identifies  areas  for  further 
development. 


PROGAM  ORGANIZATION 

The  transformation  program  consists  of 
about  100,000  source  lines  and  is  written  almost 
entirely  in  FORTRAN  or  RATF0R  (5).  It  includes 
all  the  functions  necessary  to  create  from  the 
DMA  planimetry  and  terrain  a  visual/IR  data  base 
to  be  displayed  in  real-time.  Most  processing 
is  done  in  batch  mode;  a  few  functions  depend  on 
human  judgement  or  perception  and  are  done 
interactively. 

Figure  2  shows  the  overall  organization  of 
the  transformation  program  and  indicates  the 
primary  data,  control,  and  operator  interfaces. 
The  functions  and  interfaces  will  be  described 
briefly  here  and  in  detail  in  the  following 
sections. 
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- -  DATA  FLOW 

- *-  CONTROL  FLOW 

- OPERATOR  INTERVENTION 


Display  and  evaluation  Is  not  proptijAy  part 
of  the  transformation  program,  but  is  Important  to 
the  data  base  development  process.  It  consists  of 
real-time  display  during  flight  simulation  and 
(more  Importantly)  for  visual  evaluation  of  the 
data  base.  If  errors  or  weaknesses  are  found,  the 
designer  can  repair  them  by  editing  the  planimetry 
or  terrain  or  modifying  the  transformation 
parameters  and  then  retransforming  the  affected 
areas • 


# 

*» 


r  -  Production  control  software  Includes  various 
lity  functions  to  support  and  manage  the 
production  process.  It  creates  and  maintains  data 
Dase  directories;  tracks  the  processing  of 
manuscripts  and  area  blocks;  tracks  data  base 
configuration;  reports  processing  errors;  and 
generates  a  production  log. 

DATA  BASE  ORGANIZATION 

The  data  base  can  acconmodate  up  to  100 
million  square  miles  of  visual  data  In  multiple 
disjoint  areas  anywhere  in  the  world;  Its 
organization  is  shown  in  Figure  3.  All  fixed 
features  are  assumed  to  be  between  2000  ft  below 
and  30,000  ft  above  mean  sea  level. 


Figure  2 

Overview  of  Transformation  Program 

The  process  begins  with  the  DMA  planimetry 
and  terrain  described  in  the  previous  section. 

The  data  base  desjgner  uses  interactive  analysis 
and  definition  programs  to  review  the  contents  of 
the  DMA  DDB,  define  the  visual  representation  for 
each  feature  class,  and  select  parameter  values  to 
control  the  automatic  transformation  process. 

Planimetry  processing  software  converts  the 
DMA  planimetry  to  a  standard  internal  format; 
merges  the  contents  of  adjacent  manuscripts, 
levels,  and  updates;  allows  the  designer  to  edit 
the  planimetry;  selects  a  visual/JR  model  for  each 
feature  as  directed  by  the  transformation  para¬ 
meters;  and  creates  levels  of  detail.  Planimetry 
edit  is  interactive;  all  other  steps  are  batch. 

Terrain  processing  software  almost  directly 
parallels  planimetry:  it  converts  the  DMA  terrain 
to  a  standard  internal  format;  merges  the  contents 
of  different  manuscripts;  allows  editing; 
constructs  a  polyhedral  model  of  the  terrain;  and 
creates  levels  of  detail.  All  steps  except 
editing  are  batch. 

Scene  integration  software  combines 
planimetry  and  terrain  models  into  a  unified  scene 
model:  it  divides  each  planimetry  feature  into 
pieces  contained  in  a  single  terrain  face; 
resolves  overlaps  among  features;  assigns 
elevation  values  to  the  planimetry;  builds  the 
visual  model  already  selected;  merges  planimetry 
and  terrain;  converts  the  scene  to  the  real-time 
display  format;  and  creates  the  directories  and 
linkages  necessary  to  support  real-time  retrieval. 


Figure  3 

Data  Base  Organization 

The  surface  of  the  earth  is  divided  into 
fixed  geographic  areas  called  area  blocks.  An  area 
block  is  triangular,  covers  about  25  square  miles, 
and  is  the  basic  unit  for  processing,  storage, 
and  retrieval  throughout  transformation  and 
real-time  display. 

The  data  for  each  area  block  is  divided 
into  separate  files  resulting  from  each  stage  of 
processing. 

A  section  is  a  set  of  area  blocks  which  fit 
on  a  single  disk  pack.  Sections  are  defined  by 
the  data  base  designer  and  typically  include 
200-500  area  blocks  or  5,000  to  12,500  square 
miles.  A  standard  section  holds  a  compact  set 
of  area  blocks  and  is  used  for  most  processing. 
Non-standard  sections  hold  input  planimetry  and 
terrain  until  they  can  be  reblocked  into 
standard  sections. 
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ANALYSIS  AND  DEFINITION 

Analysis  and  definition  software  includes 
programs  which  the  data  base  designer  uses  to 
review  and  analyze  the  contents  of  the  DMA  DOB, 
to  define  the  visual /IR  representation  of 
feature  classes,  and  to  create  command  files 
which  control  the  transformation  process. 

Analysis  software  provides  the  capability 
to  list  planimetry  feature  attributes  and 
coordinates;  list  terrain  elevations;  plot 
planimetry  or  terrain;  and  analyze  the 
statistical  distribution  of  feature  type,  size, 
or  attribute  values.  Analyses  may  be  done  for 
all  features  in  a  manuscript  or  for  classes  of 
features  defined  by  type,  position,  or  attribute 
values. 

The  planimetry  conversion  table  is  created 
by  the  designer  and  defines  the  desired  visual 
and  IR  representation  for  planimetry  features. 

It  is  indexed  by  FID,  SMC,  and  type  and 
specifies  the  desired  model  class,  specific 
library  or  parameterized  model,  color,  IR  class, 
significance  (relative  accuracy  requirement!, 
and  preference  (relative  inclusion  priority). 

Four  classes  of  models  are  provided,  as 
shown  in  Figure  4:  a  library  object  is  modelled 
by  hand  prior  to  transformation  and  can  be 
inserted  anywhere;  it  may  be  as  complex  as 
desired.  Parameterized  objects  are  less  complex 
and  are  built  automatically  by software  to  a 
specified  height,  length,  width,  and 
orientation.  Carpets  are  still  less  complex  and 
are  built  automatically  to  a  specified  height 
and  outline.  Paint-ons  are  simply  colored 
outlines  laid  directly  on  the  terrain.  Finally, 
a  feature  may  be  omitted  completely. 
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Figure  4 
Visual  Models 


The  IR  class  describes  the  characteristic 
amplitude  and  lag  of  thermal  variations  relative 
to  solar  illumination;  it  is  used  to  simulate 
the  diurnal  variation  of  infrared  sensor 
imagery. 

Significance  identifies  features  such  as 
navigation  landmarks  for  which  the  correct  shape 
is  important;  preference  identifies  features 
which  are  important  to  include  in  the  scene.  If 
the  scene  must  be  simplified  to  match  DIG 
capacity,  these  features  will  be  the  last  to  be 
simplified  or  discarded. 

Command  files  are  created  by  the  designer 
to  control  transformation  and  are  entered  as  a 
list  of  free- form  commands  and  parameter 
assignment  statements.  The  parameters  define  the 
planimetry  conversion  table  to  be  used,  number  of 
levels  of  detail  to  be  built,  edge  and  error 
budgets  for  each  level ,  the  target  DIG,  and 
various  special  parameters. 

PLANIMETRY  PROCESSING 

Planimetry  processing  is  shown  in  Figure  5, 
which  traces  the  processing  steps  that  convert  the 
DMA  planimetry  into  a  visual  model.  Figure  6 
illustrates  the  output  from  three  key  steps. 
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Figure  5 

Planimetry  Processing 

Planimetry  preparation  software  reads 
selected  manuscripts  from  magnetic  tape,  verifies 
that  all  features  are  in  proper  format,  and 
converts  it  to  an  internal  format  designed  to 
facilitate  editing  and  transformation.  It  also 
divides  the  manuscript  into  area  blocks  and 
allocates  features  to  the  area  blocks  they 
overlap.  The  points  where  a  feature  crosses  an 
area  block  boundary  are  marked  to  assure  con¬ 
tinuity  even  though  area  blocks  are  processed 
independently.  Finally,  it  merges  the  manuscripts 
and  levels  rrfilch  overlap  each  area  block.  If 
manuscripts  overlap  each  other,  the  designer  must 
select  one  to  define  the  overlap  region. 
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Prepared  and  Edited 
Planimetry 


Figure  6 

Key  Planimetry  Processing  Steps 

Planimetry  editing  allows  the  designer  to 
Insert  features  not  Included  In  the  DMA  DDB  or 
correct  any  errors.  The  designer  can  select 
features  by  identifier,  position  or  attributes; 
print  or  modify  feature  attributes;  plot  or 
digitize  feature  coordinates;  and  create  or  delete 
features.  The  program  uses  a  Tektronix  4014 
graphics  terminal  for  graphical  Interaction. 


The  prepared  and  edited  planimetry 
(composite  planimetry)  Is  retained  to  permit  DMA 
updates,  further  editing,  or  retransformation  with 
new  parameters. 

Planimetry  modelling  selects  a  vIsual/IR 
model  for  each  feature,  using  Its  feature 
identification  and  surface  material  code  to  Index 
Into  the  planimetry  conversion  table.  It  also 
identifies  and  passes  to  terrain  processing  hydro- 
graphic  features  (lakes,  rivers,  etc.)  which  must 
be  correlated  with  terrain  modelling. 

Planimetry  levelling  reduces  the  scene 
complexity  to  match  the  display  capacity  of  the 
DIG  and  produces  several  levels  of  detail  to  use 
that  capacity  most  effectively:  the  most-detailed 
version  will  be  displayed  near  the  viewer  and 
less-detailed  versions  In  the  distance.  The 
complexity  is  reduced  by  removing  Insignificant 
features;  degrading  the  representation  from 
library  object  to  parameterized  object  to  carpet 
to  paint-on;  or  by  simplifying  the  feature 
outline.  Features  marked  as  significant  or 
preferred  will  be  the  last  to  be  simplified  or 
discarded. 

TERRAIN  PROCESSING 

Terrain  processing  is  Illustrated  in  Figure 
7,  tracing  the  conversion  of  the  DMA  terrain  Into 
a  visual  model. 


Figure  7 

Terrain  Processing 

Terrain  preparation  reads  selected 

manuscripts  from  magnetic  tape,  verifies  that  they 
contain  no  format  errors,  and  converts  them  to  an 
Internal  format.  It  divides  the  manuscript  Into 
area  blocks  and  allocates  elevation  grldposts  to 
the  area  blocks  they  lie  In.  Grldposts  on  or 
near  an  area  block  boundary  are  saved  In  all 
adjacent  area  blocks  to  assure  continuity. 
Finally,  it  merges  the  manuscripts  and  levels 
which  overlap  each  area  block.  If  two 

manuscripts  overlap,  the  designer  must  select 
the  dominant  one. 
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Terrain  editing  allows  the  designer  to 
Inspect  or  modify  terrain  elevations  before 
transformation.  He  may  plot  perspective  views 
of  the  terrain  and  print  or  modify  elevation 
posts. 

The  prepared  and  edited  terrain 
(composite  terrain)  Is  retained  to  permit  DMA 
updates,  further  editing,  or  retransformation 
with  new  parameters. 

Terrain  modelling  builds  a  sequence  of 
visual  models  by  successive  approximation  to  the 
given  elevations  as  shown  in  Figure  8.  Each 
model  Is  a  continuous  polyhedral  sheet  of 
triangular  faces;  vertex  elevations  are 
determined  by  interpolation  in  the  given 
elevations.  The  first  model  is  the  plane 
determined  by  the  three  vertices  of  the  area 
block.  This  triangle  Is  recursively  subdivided 
into  two  smaller  triangles  by  a  vertical  cutting 
plane  through  one  vertex;  the  elevation  of  the 
new  vertex  created  Is  determined  by 
interpolation  in  the  given  elevations.  If  the 
new  vertex  is  not  on  the  edge  of  the  area  block, 
a  second  cutting  plane  is  added  from  the  new 
vertex  to  the  opposite  vertex  of  the  adjacent 
triangle. 


First  Terrain  Model 


•UP 


Second  Terrain  Model 


Third  Terrain  Model 


Any  of  several  heuristics  are  used  to 
select  at  each  step  the  triangle  to  subdivide 
and  the  cutting  plane  to  use.  Several 
conflicting  requirements  must  be  balanced:  the 
model  should  match  the  terrain  as  closely  as 
possible  for  each  number  of  faces;  the  model 
must  correspond  correctly  to  hydrographic 
features;  excessively  thin  triangles  should 
not  be  generated;  and  models  must  be  continuous 
across  area  block  boundaries. 

Terrain  levelling  selects  from  this 
sequence  of  approximations  a  set  of  models  which 
match  the  display  capacity  of  the  DIG.  As  with 
planimetry,  the  most-detailed  version  is  used 
near  the  viewer  and  less-detailed  versions  in 
the  distance. 


SCENE  INTEGRATION 

Scene  Integration  is  diagrammed  in  Figure 
9.  Figure  10  illustrates  the  output  from  two 
key  processing  steps.  Each  level  of  detail  is 
integrated  in  turn. 

Planimetry  division  divides  each  planimetry 
feature  into  pieces,  eact  of  which  is  contained 
on  a  single  terrain  face.  library  and 
parameterized  objects  are  not  exempt:  they  must 
be  divided  to  support  real-time  occulting. 
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Figure  9 

Scene  Integration 


Built  Planimetry 


Overlap  resolution  tests  each  terrain  face 
for  overlaps  among  the  feature  pieces  lying  on 
that  face.  If  any  overlaps  are  found,  the  feature 
with  the  highest  FAC  #  has  priority.  Depending  on 
the  visual  representation  of  the  features 
involved,  the  masked  portion  of  the  lower  feature 
may  be  removed,  the  lower  feature  may  be  assigned 
a  lower  occulting  priority,  or  the  higher 
feature  may  be  elevated  to  rest  on  the  lower. 

Terrain  alignment  sets  the  elevation  of  each 
feature  piece  so  that  it  rests  on  the  terrain 
model.  Each  vertex  of  a  paint-on  or  base  vertex 
of  a  carpet  is  set  to  the  terrain  model  elevation 
at  that  location;  thus  a  paint-on  or  carpet  will 
slope  to  follow  the  terrain.  A  library  or  para¬ 
meterized  object  is  elevated  until  the  center  of 
its  base  coincides  with  the  terrain;  it  remains 
horizontal  and  any  part  below  the  surface  is 
removed . 

Model  building  creates  the  visual  models 
selected  in  planimetry  processing:  library  objects 
are  copied  into  the  scene;  parameterized  objects 
are  built  to  the  specified  parameters;  and  carpets 
are  raised  above  the  terrain  and  provided  with 
sidewalls  of  the  specified  height.  Paint-ons 
require  no  construction.  All  planimetry  models 
are  added  to  the  terrain  model . 

Reformatting  converts  the  completed  scene 
into  ~i  format  designed  to  support  real-time 
display  and  to  use  minimum  file  space.  In 
addition,  it  computes  several  attributes  and  data 
structures  needed  to  improve  real-time  efficiency 
and  capacity. 

Mission  building  selects  the  area  blocks 
required  to  support  a  selected  flight  path  and 
builds  the  directory  structure  needed  for 
real-time  retrieval.  A  typical  mission  will  occupy 
several  300MB  disk  packs.  Two  packs  are  on  line 
at  any  given  time;  as  the  aircraft  flies  they  are 
alternately  exchanged  for  other  packs  containing 
the  new  areas  entered. 

DISPLAY  AND  EVALUATION 

Display,  evaluation,  and  revision  are 
illustrated  in  Figure  11.  The  visual  scene  is 
displayed  on  a  test  monitor,  out  the  cockpit 
windows,  or  on  the  EVS  monitor  in  the  cockpit. 
The  display  is  normally  integrated  with  flight 
simulation  and  is  correlated  with  all  other 
simulated  systems.  Stand-alone  operation  is  also 
provided;  scene  viewpoint  and  display  parameters 
are  controlled  at  a  visual  system  diagnostic 
control  panel . 

The  display  is  evaluated  visually,  if  it  is 
not  satisfactory,  corrections  are  made  by 
adjusting  the  transformation  parameters,  editing 
the  planimetry,  or  editing  the  terrain.  Then  the 
area  blocks  which  have  been  modified  must  be 
retransformed;  the  rest  are  unchanged. 


Integrated  Scene 
Figure  10 

Key  Scene  Integration  Steps 
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Display  t  Evaluation 
PRODUCTION  CONTROL 

This  program  is  used  to  produce  very  large 
data  bases;  management,  production  logistics, 
and  configuration  control  required  careful 
attention. 

Except  for  edit  programs  and  visual 
evaluation,  the  transformation  program  operates  in 
batch  mode.  The  data  base  designer  prepares 
command  files  which  specify  the  area  blocks  to  be 
transformed,  the  process  steps  to  be  done,  and  the 
parameters  which  will  control  the  result.  Once 
started,  the  program  requires  operator 
Intervention  only  to  mount  tapes  or  change  disk 
packs. 

The  program  Includes  comprehensive  checks 
on  the  consistency  of  input  and  Intermediate 
files  and  on  Its  own  operations;  If  any  check  is 
not  satisfied,  an  error  message  is  printed  out. 
Warning  messages  report  conditions  which  are 
legal  but  suspicious;  fatal  errors  report 
illegal  conditions.  If  the  effect  of  a  fatal 
error  is  limited  to  a  single  feature  or  area 
block,  the  program  will  bypass  that  feature  or 
area  block  and  continue;  only  If  continuation  is 
impossible  or  risks  damage  to  permanent  files 
will  the  program  stop.  The  error  message  will 
describe  the  error  and  will  Identify  the 
specific  erroneous  area  block  or  feature. 

Each  production  run  is  automatically 
assigned  a  unique  Production  Control  Number 
(PCN)  and  prints  out  a  production  log  as  a 
permanent  historical  summary  of  that  run.  The 
production  log  identifies  the  PCN  of  the  run, 
the  program  version,  the  value  of  each  Input 
parameter,  any  error  messages,  run  statistics, 
and  the  name  and  PCN  of  each  input  file. 

Complete  tracking  of  all  Intermediate  and 
output  files  Is  provided.  Each  file  contains 
the  PCN  of  the  production  run  which  created  or 
last  updated  It.  The  corresponding  production 
log  lists  all  relevant  Information,  Including 
the  name  and  PCN  of  each  input  file.  Then  the 
production  log  for  each  Input  can  be  examined, 
and  so  on  back  to  the  original  DMA  manuscript. 


In  addition,  the  composite  planimetry  and 
terrain  files  Identify  the  DMA  manuscripts 
Included  in  each  area  block  and  the  area  that 
each  manuscript  covers;  each  feature  can  be 
traced  back  to  the  original  DMA  manuscript  and 
feature. 


SIGNIFICANT  ADVANCES 

The  program  described  here  has  demonstrated 
several  significant  advances  over  previous 
techniques  for  the  construction  of  visual  data 
bases. 

Utilization  of  the  DMA  DDB:  The  DMA  DDB 
covers  most  of  the  United  States  and  parts  of 
the  rest  of  the  world,  and  is  intended  for 
tactical  planning  and  radar  simulation.  The 
transformation  program  makes  this  data  base 
available  for  visual  and  electro-optical 
simulation  and  facilitates  correlation  with 
other  sensors  based  on  the  same  DDB. 

Automatic  terrain  modelling:  The  DMA  DDB 
describes  the  terrain  with  high  resolution.  The 
transformation  program  builds  a  model  of  this 
terrain  which  can  be  displayed  in  real-time. 

The  algorithm  used  can  easily  be  extended  to  use 
other  regular  or  Irregular  elevation  grids.  The 
need  for  specially-trained  data  base  modellers 
is  reduced. 

Parameterized  and  library  objects:  The  DMA 
DDB  specifies  little  about  the  actual  shape  of  a 
point  feature.  Parameterized  objects  provide 
the  automatic  construction  of  objects  that  meet 
the  given  length,  height,  and  width.  A 
selection  of  generic  shapes  allows  the 
differentiation  and  visual  identification  of 
different  feature  classes.  When  a  feature  needs 
to  be  more  complex,  library  objects  allow  the 
insertion  of  3D  models  designed  manually. 

Automatic  level  of  detail  generation; 
Levelling  improves  apparent  scene  content by 
removing  unresolvable  or  less  resolvable  detail 
from  the  background  and  adding  more  detail  to  the 
foreground.  The  transformation  program 
automatically  generates  levels  of  detail  for  both 
planimetry  and  terrain. 

Planimetry/terrain  integration;  Planimetry  is 
normally  mapped  as  if  it  were  projected  onto  mean 
sea  level,  and  is  conveniently  processed  in  this 
form;  the  terrain  is  processed  Independently.  The 
transformation  program  automatically  integrates 
the  planimetry  and  terrain  models  Into  a  scene 
model.  This  technique  separates  and  simplifies 
the  processing  of  planimetry  and  terrain  without 
sacrificing  the  ability  to  create  an  integrated 
scene. 
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Large  area  update:  DMA  Issues  updated 
manuscripts  from  time  to  time.  The  transformation 
program  can  read  these  new  manuscripts  and 
retransform  the  modified  area  blocks  without 
affecting  the  rest.  This  allows  the  data  base  to 
be  kept  current  with  minimum  effort. 

World-wide  transformation:  The  data  base 
coordinate  systems,  directories,  and  structure 
have  been  designed  so  that  the  transformation 
program  can  process  data  anywhere  in  the  world.  A 
round  earth  model  provides  good  correlation  with 
navigational  systems.  In  addition,  the  data  base 
is  designed  to  accommodate  the  processing, 
retention,  and  use  of  data  bases  containing  up  to 
100  million  square  miles. 

Radical  improvement  in  productivity:  The 
transformation  program  can  generate  fully-debugged 
data  bases  about  100  times  faster  than  previous 
manual  techniques,  depending  on  scene  complexity 
and  degree  of  manual  enhancement.  Furthermore, 
automatic  transformation  is  still  in  its  infancy, 
and  significant  further  improvements  can  be 
expected. 

Configuration  tracking:  A  data  base  will  be 
transformed  from  many  DMA  manuscripts,  involve 
many  sets  of  transformation  parameters,  and  will 
gradually  evolve  as  updated  manuscripts  and  manual 
enhancements  are  incorporated.  Every  file  can  be 
traced  back  through  the  production  log,  ultimately 
identifying  each  original  manuscript,  intermediate 
file,  program  run,  and  parameter  value  involved  in 
the  creation  of  that  file.  In  addition,  each 
planimetry  feature  can  be  traced  to  the  original 
DMA  manuscript  containing  it.  These  techniques 
provide  a  permanent  record  of  how  a  data  base  was 
built  and  facilitate  the  diagnosis  and 
backtracking  of  errors. 

FURTHER  DEVELOPMENT 

The  transformation  program  provides  a  new 
order  of  capability;  however,  it  also  reveals  a 
few  areas  in  which  it  is  not  yet  known  how  to 
effectively  use  these  capabilities  or  *4\ere  the 
results  are  not  yet  as  good  as  manual  techniques. 
LINK  has  already  begun  further  development  of  the 
transformation  program  to  realize  the  full 
potential  of  automatic  transformation. 

Terrain  fidelity;  The  present  terrain 
modelling  algorithm  is  controlled  by  the  elevation 
error  between  the  input  and  the  model.  As  a 
result,  it  tends  to  build  an  accurate  model 
(within  the  edge  budget)  but  often  fails  to 
reproduce  the  visual  appearance  (or  fidelity). 
Since  many  pilot  tasks  seem  to  depend  more 
heavily  on  terrain  fidelity  than  on  elevation 
accuracy,  more  work  is  being  done  to  identify 
the  determinants  of  terrain  fidelity  and  to 
develop  algorithms  that  preserve  fidelity. 


DMA  Planimetry:  The  DMA  DDB  was  originally 
designed  to  support  radar  landmass  simulation. 

The  terrain  DDB  is  readily  usable  for  visual  or 
electro-optical  simulation,  but  the  planimetry 
DDB  has  definite  limitations.  First,  it 
contains  little  information  on  3D  shape,  color, 
or  IR  characteristics.  Second,  it  often  omits 
features  such  as  roads  or  rivers  which  are 
important  visually  but  not  on  radar.  Third,  in 
many  areas  it  does  not  contain  sufficient 
features  to  support  low-level  flight.  Fourth, 
the  data  is  sometimes  old  and  does  not  Include 
significant  features  built  recently.  The 
transformation  program  cannot  solve  these 
problems,  but  work  is  being  done  to  Improve  its 
edit  and  analysis  capabilities  and  to 
incorporate  data  from  other  sources  into  the 
final  scene. 

Manual  enhancement:  The  present 
transformation  program  does  not  provide  for  the 
insertion  of  entirely  hand-modelled  areas  such  as 
airports  and  limits  manual  enhancement  of  the 
transformed  scene.  Since  DMA  does  not  provide 
adequate  detail  to  support  take-off/landing 
scenes  or  tactical  air-to-ground  scenes,  these 
must  be  entirely  hand-modelled  or 
manually-enhanced  transformed  scenes.  More  work 
is  being  done  to  develop  the  capability  to 
combine  automatic  and  hand  modelling  and  to  take 
full  advantage  of  both. 
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ABSTRACT 


Military  requirements  for  large  scale  training  missions  on  computer  image  generation 
(CIG)  systems  have  placed  increased  emphasis  on  the  CIG  data  base  development  process. 

General  Electric  produces  large  scale  data  bases  in  a  semi-automatic  process  involving  the 
transformation  and  enhancement  of  Defense  Mapping  Agency  digital  data  bases  into  CIG  scene 
descriptors.  The  enhancement  process  is  the  limiting  factor  in  the  evolution  of  a  fully  auto¬ 
mated  data  base  development  system,  and  suggests  a  requirement  for  a  single  source  data  base. 
Future  General  Electric  CIG  systems  will  incorporate  automation  technology  to  allow  for  low 
cost  generation  of  visual  scenes  meeting  strategic  applications  of  specific  weapon  systems 
trainers. 


INTRODUCTION 

The  emergence  of  computer  image  generation  (CIG) 
in  aircrew  training  as  a  multimillion  dollar  indus¬ 
try  is  primarily  attributable  to  advances  in  elec¬ 
tronic  technology.  Recently,  however,  CIG  manufac¬ 
turers  have  diverted  research  and  development  ac¬ 
tivities  from  computer  hardware  improvements  to  an 
increasing  emphasis  on  the  development  of  sophisti¬ 
cated  techniques  used  in  the  generation  of  mathe¬ 
matical  descriptors  supporting  simulation  displays. 
These  CIG  data  bases  supply  the  necessary  scene 
content  constituting  such  cockpi  t  panoramas  as  radar, 
forward  looking  infrared  radar  ( FUR) ,  low  light  tele¬ 
vision  (LLTV) ,  and  "out  the  window"  visuals. 

While  early  simulators  provided  visual  cues  for 
aircrew  training  in  various  airport  (and  aircraft 
carrier)  maneuvers,  more  recent  simulator  systems 
have  undertaken  the  portrayal  of  large  geographic 
areas  containing  sufficient  scene  detail  to  support 
low  altitude  visual  navigation.  The  ultimate  goal 
of  this  development  trend  appears  to  be  the  manu¬ 
facture  of  aircraft  simulators  capable  of  global 
flight  paths  arbitrarily  selected  by  the  flight 
instructor. 

The  complexity  arising  from  the  real  time  re¬ 
trieval  and  display  of  high  detail  scenes  has  placed 
an  enormous  burden  on  the  design  of  CIG  hardware, 
and  has  forced  most  manufacturers  to  combine  large 
core  memory  storage  with  high  speed  parallel  pro¬ 
cessing  in  order  to  generate  adequate  cockpit  dis¬ 
plays.  0)  In  most  cases,  however,  the  industry  has 
overlooked  problems  associated  with  preparation  of 
the  CIG  data  base  itself.  While  data  bases  inearly 
CIG  systems  were  prepared  in  the  tedious  fashion 
described  by  Sutherland, (2)  experience  has  shown  that 
the  manual  preparation  of  large  scale  data  bases  is 
both  expensive  and  impractical.  It  seems  evident 
that  overall  progress  in  the  CIG  technology  now 
relies  on  the  total  automation  of  the  data  base 
development  process. 


HISTORICAL  BACKGROUND 

General  Electric's  Electronic  Systems  Labora¬ 
tory  developed  the  first  computer  image  generator  in 
1958.  This  primitive  device  produced  a  regular  line 
pattern  over  a  ground  plane  to  create  the  illusion 
of  motion,  but  was  clearly  inadequate  as  a  practical 
training  tool.  Subsequent  CIG  systems  produced  by 
General  Electric  and  other  manufacturers  have  shown 
significant  progress  in  the  enhancement  of  detail  in 
the  CIG  displays. 

Scene  detail,  in  the  language  of  the  industry, 
is  measurable  by  counting  the  number  of  visible 
edges  displayed  by  the  CIG  at  a  given  instant  during 
real  time  operation.  As  is  illustrated  in  figure  1, 
the  growth  in  the  number  of  displayed  edges  in  var¬ 
ious  General  Electric  systems  has  been  virtually 
exponential  over  the  last  12  years.  It  is  no  coin¬ 
cidence  that  thi  s  growth  curve  paral  1  el s  the  exponen¬ 
tial  miniaturization  of  computer  components  aris¬ 
ing  from  technological  advances  in  medium  and  large 
scale  electronic  integration.  The  growth  in  com¬ 
plexity  of  CIG  data  bases,  on  the  other  hand,  is 
small  by  comparison.  In  fact,  figure  2  indicates 
the  number  of  environment  edges  contained  in  General 
Electric  CIG  data  bases  actually  decl ined  after  1974. 


The  disparity  apparent  in  the  evolution  of  CIG 
hardware  versus  CIG  data  bases  arises  from  the  mod¬ 
est  applications  around  which  early  CIG  devices 
were  designed.  These  products  were  intended  pri¬ 
marily  for  use  in  landing/take-off  maneuvers,  and  as 
such,  required  only  a  single  airport  complex  in  the 
data  base  environment.  More  recent  projects  have 
reflected  a  change  in  this  application  philosophy, 
and  have  encouraged  expansion  of  flight  scenarios  to 
include  air-to-air  combat,  in-flight  refueling  and 
low  altitude  navigation.  One  current  aircraft  sim¬ 
ulation,  for  example,  will  require  a  supporting  data 
base  portraying  a  geographic  area  of  40.000  square 
miles.  The  generation  of  a  data  base  of  this 
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Figure  2.  Growth  in  General  Electric  CIG 
Data  Base  Capacities' 1 


magnitude  using  traditional  manual  techniques  would 
require  thousands  of  man  years  to  produce  at  a  cost 
of  hundreds  of  millions  of  dollars  to  the  product 
consumer. 


AUTOMATED  DATA  BASE  GENERATION 

In  1977  General  Electric  initiated  the  develop¬ 
ment  of  a  computer  program  capable  of  generating 
CIG  data  bases  with  a  minimum  of  human  interaction. 
As  shown  in  figure  3,  inputs  to  this  program  are 
supplied  by  the  Defense  Mapping  Agency  Aerospace 
Center  (DMAAC)  in  the  form  of  digital  terrain  and 
culture  data  bases.  DMAAC  terrain  files  consist  of 
elevation  values  sampled  at  regularly  spaced  inter¬ 
vals  over  the  surface  of  the  earth.  The  raw  grid 
data  is  first  blocked  into  geographic  areas  of 
manageable  size,  and  a  simplified  subset  is  then 
obtained  by  detecting  points  of  significance  in  the 
topological  environment  such  as  mountains,  valleys, 
and  ridge  lines.  Finally,  a  triangular  faceted 
surface  is  fitted  to  the  selected  elevation  values 
to  force  an  approximation  of  rolling  terrain. 


Figure  3.  CIG  Data  Base  Development  Using 
DMAAC  Digital  Data 

Planimetric  features  such  as  lakes,  forests, 
and  highways  are  also  derived  from  DMAAC  digital 
data  bases.  After  blocking  the  raw  input  data  in 
correspondence  with  the  underi' inq  terrain,  DMAAC 
cultural  features  ."’i  process-?--,  :.v  the  degree  of 
simplicity  req«  Vt,  -,y  the  CIG.  li.e  resulting  cul¬ 
ture  data  is  then  combined  with  the  underlying  tri¬ 
angulated  terrain  to  form  a  single  surface  file. 

Data  Base  Enhancement 


Although  DMAAC  data  bases  contain  more  infor¬ 
mation  than  the  CIG can  process,  nevertheless,  cer¬ 
tain  cultural  features  used  by  aircrews  as  naviga¬ 
tional  cues  are  often  absent  from  the  DMAAC  cultural 
data  bases.  Therefore,  the  enhancement  of  the 
original  DMAAC  data  using  supplemental  inputs  such 
as  Joint  Operations  Graphic  (JOG)  and  U.S.  Geologi¬ 
cal  Survey  (USGS)  Quandrangle  charts  is  a  necessity. 

General  Electric  presently  employs  a  turnkey 
interactive  graphics  system  to  digitize  data  from 
JOG  and  USGS  inputs.  The  digitized  data  from  this 
process  must  be  formatted  to  match  DMAAC  culture 
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Sun  Shaded  DMAAC  Terrain  Elevation 
Data  (Sun  Angle  =  45°) 


i.  Unprocessed  DMAAC  Culture  Data 
(Surface  Material  Categories 
are  Color  Encoded) 


Figure  7.  DMAAC  Culture  Data  Enhanced  with 
Manually  Digitized  JOG  Data 


Figure  8.  Enhanced  DMAAC  Culture  Data  Merged 
with  DMAAC  Terrain  Elevation  Data 


DMAAC  -  JOG  Data  from  Manual 
Digitization  (Roads,  Railroads 
and  Waterways) 


Figure  9.  Generation  of  Three  Dimensional 
Image  from  Stereopairs 


file  specifications,  and  the  two  files  must  be 
merged  to  form  a  single  input  to  the  automated  data 
base  generation  program.  Figures  4  through  8  illus¬ 
trate  the  output  of  various  steps  of  the  culture 
enhancement  process. 

Traditional  manual  methods  of  data  preparation 
are  still  required  as  a  final  step  in  the  data  base 
generation  process.  In  addition  to  certain  plani- 
metric  features  not  contained  in  DMAAC  or  supple¬ 
mental  data,  models  such  asbuildings,  dams,  bridges, 
and  aircraft  are  painstakingly  drafted  and  posi¬ 
tioned  in  the  final  CIS  data  base.  This  final  step 
is  responsible  for  a  large  portion  of  data  base 
development  costs. 

FUTURE  DIRECTIONS  IN  AUTOMATION 

Manual  enhancement  of  CI6  data  bases  necessi¬ 
tated  by  Defense  Mapping  Agency  data  deficiencies 
is  the  limiting  factor  in  total  automation  of  the 
CIG  data  base  development  process.  A  joint  effort 
of  commercial  and  government  agencies  might  someday 
lead  to  the  production  of  a  single  source  data  base 
containing  sufficient  detail  for  all  CIG  visual 
data  bases.  In  the  meantime,  at  least  two  possi¬ 
bilities  exist  for  the  improvement  of  CIG  data  base 
quality  which  may  also  lead  to  lower  system  costs. 

Automatic  Digitization 

Cultural  enhancement  of  CIG  visual  data  bases 
involves  the  extraction  of  planimetric  data  from 
various  maps,  charts,  and  photographs  using  manual 
digitization  methods,  and  the  human  operator  must 
make  decisions  regarding  the  relative  importance  of 
features  that  are  candidates  for  inclusion  in  the 
final  data  base.  The  lack  of  selection  criteria, 
combined  with  errors  arising  from  eye-hand  coordi¬ 
nation  and  operator  fatigue,  make  manual  digitiza¬ 
tion  an  inherently  error  prone  and  costly  process. 

Alternatives  to  manual  digitization  have  been 
of  interest  to  cartographers  for  many  years,  and 
devices  such  as  facsimile  and  Brush  recorders  are 
routinely  used  in  the  preparation  of  maps  and 
charts.  Unfortunately,  these  products  are  usually 
designed  to  generate  raster-formatted  data  which  is 
incompatible  with  the  vector- formatted  data  re¬ 
quired  by  most  CIG  systems.  Also,  facsimile  and 
Brush  devices  are  extremely  slow,  and  in  general, 
the  data  output  from  these  systems  is  too  dense  for 
use  in  flight  simulation  data  bases.  Recently, 
Boyle  reported  the  development  of  an  automatic 
digitization  system  using  a  flying  spot  scan  of  map 
separations.  W  This  device  is  of  potential  value  to 
CIG  data  base  development  systems  by  virtue  of  the 
vector-formatted  output  generated  by  the  product, 
and  the  high  speed  with  which  data  is  captured. 
Furthermore,  the  use  of  map  separations  lends  itself 
to  the  segregation  of  cultural  data  into  logical 
groups  such  as  water,  forests,  and  highways  for 
enhancement  of  DMA  digital  data  bases. 

Stereometric  Photography 

Photogrammetry  is  the  science  of  detecting  in¬ 
formation  about  physical  objects  by  recording, 
measuring,  and  interpreting  photographic  images.  As 
with  automatic  digitization,  photogrammetry  has 
been  used  for  some  time  by  government  and  commercial 
cartographers  for  the  preparation  of  maps  and 
charts.  Metric  photogrammetry  involves  the  process¬ 
ing  of  overlapped  photographs  known  as  stereopairs 


to  extract  accurate  three  dimensional  information 
about  real  world  objects.  The  use  of  translucent 
positives,  or  diapositives,  to  form  a  three  dimen¬ 
sional  image  is  illustrated  in  figure  9. 

The  potential  application  of  stereometric 
photography  to  CIG  data  base  development  was  demon¬ 
strated  recently  for  the  Naval  Training  Equipment 
Center  in  Orlando,  Florida.  In  this  experiment, 
a  Russian  ship  was  digitized  from  stereophotographs 
taken  of  a  scale  model  using  close  range  camera 
equipment.  As  shown  in  the  calligraphic  plot  in 
figure  10,  the  350  polygon  feature  contains  more 
than  sufficient  detail  to  be  very  effective  in  a 
CIG  visual  simulation,  and  furthermore,  the  feature 
was  prepared  in  just  two  days  using  photogramnetric 
methods.  The  preparation  of  a  similar  feature 
utilizing  traditional  CIG  data  base  development 
techniques  would  consume  several  months  of  effort. 

CONCLUSION 

Visual  scenarios  now  demanded  of  CIG  flight 
simulators  have  forced  the  re-evaluation  of  tradi¬ 
tional  data  base  creation  methods.  General  Electric 
has  undertaken  the  automation  of  the  data  base 
generation  process,  and  is  committed  to  providing 
low  cost  simulators  that  will  meet  future  consumer 
requirements.  It  is  evident  that  government  spon¬ 
sored  research  should  be  aimed  at  the  acquisition 
of  high  technology  CIG  data  base  generation  tech¬ 
niques,  such  as  metric  photogrammetry  and  auto¬ 
matic  scan  digitization,  which  will  lead  to  lower 
costs  in  future  CIG  systems. 
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Figure  10.  Russian  Ship  Digitized  from  Stereometric  Photographs 
(Computer  Systems  laboratory,  NTEC,  Orlando,  Florida) 
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ABSTRACT 

Aliasing  in  computer  generated  images  produces  artifacts  which  degrade  training  effective¬ 
ness.  A  rigorous  implementation  of  low-pass  filtering  required  to  prevent  aliasing  requires 
computation  too  extensive  to  be  incorporated  into  real-time  computer  image  generation  (CIG) 
systems.  As  a  result,  current  CIG  systems  employ  poor  approximations  to  proper  filtering, 
and  aliasing  still  occurs. 

This  paper  discusses  the  theory  of  image  filtering  and  demonstrate  a  new  real-time  anti¬ 
aliasing  technique  developed  from  the  theory.  The  new  technique  represents  a  much  closer 
approximation  to  the  rigorous  solution  and  therefore  produces  images  of  much  higher  quality 
than  current  real-time  techniques.  At  the  same  time  it  requires  less  computation. 


INTRODUCTION 

Computer  Image  Generation  systems  represent 
a  continuously  varying  image  by  limited  sampling 
in  both  time  and  space.  The  dynamic  image  is 
represented  as  a  sequence  of  static  images  or 
frames  sampled  in  time  at  a  rate  between  30  and 
60  frames  per  second.  Each  frame  is  represented 
by  a  finite  number  of  spatially  sampled  picture 
elements  or  pixels  generally  arranged  in  a  raster 
format  with  equal  spacing.  The  Nyquist  Sampling 
Theorem  states  that  for  any  sampled  signal  any 
frequency  greater  than  one  half  the  sampling 
frequency  will  not  be  properly  represented.  The 
result  is  called  aliasing  because  the  under¬ 
sampled  frequency  components  appear  as  different 
lower  frequencies.  Aliasing  is  a  critical 
problem  in  CIG  displays  for  training  because  it 
produces  artifacts  which  are  distracting  and 
confusing  to  the  trainee.  Temporal  aliasing 
occurs  when  the  dynamic  image  changes  signifi¬ 
cantly  between  sampled  frames.  Spatial  aliasing 
occurs  when  any  frame  contains  spatial  frequen¬ 
cies  greater  than  one  half  the  spatial  sampling 
frequency.  In  general,  temporal  aliasing  occurs 
only  during  extreme  vehicle  maneuvers  such  as 
high  speed  rotation.  Spatial  aliasing,  however, 
occurs  in  every  frame  containing  high  frequency 
information,  such  as  that  inherent  in 
well-defined  boundaries  between  surfaces.  In 
addition,  spatial  aliasing  anomalies  are  aggra¬ 
vated  by  dynamic  presentation  of  the  image 
sequence  because  each  successive  image  is  sampled 
slightly  differently.  As  a  result  crawling, 
jumping  and  scintillation  effects  are  produced  at 
surface  boundaries.  Because  the  eye  is  so  sensi¬ 
tive  to  dynamic  changes,  the  temporal  effects  of 
spatial  aliasing  are  extremely  distracting  and 
therefore  degrade  the  training  effectiveness  of 
the  CIG  system.  At  the  present  time,  therefore, 
spatial  aliasing  is  by  far  the  mere  critical 
problem  to  be  solved  to  improve  image  quality. 

Consequently,  we  have  concentrated  our  study  on 
developing  an  effective  technique  for  spatial 
antial iasing. 

In  seeking  a  solution  to  the  spatial  anti¬ 
aliasing  problem,  we  must  consider  the  frequency 
content  of  the  images  we  will  be  generating  and 
be  careful  to  sample  at  a  frequency  at  least 
twice  that  of  the  highest  frequency  in  a  given 
image.  However,  since  sharp  surface  boundaries 


and  corners,  which  contain  infinite  frequency 
components,  are  common,  any  finite  sampling  of 
the  unfiltered  image  will  produce  aliasing.  Con¬ 
strained  by  a  system  with  finite  sampling,  the 
ideal  solution  is  to  prefilter  the  image  before 
sampling  to  remove  all  frequencies  higher  than 
half  the  sampling  frequency.  This  can  be  done  in 
two  ways.  First,  the  Fourier  transform  of  the 
image  can  be  computed  and  multiplied  by  the  fre¬ 
quency  domain  transfer  function  of  the  appro¬ 
priate  low-pass  filter.  The  inverse  transform  of 
the  product  will  produce  the  filtered  image. 
Alternatively,  we  could  perform  the  equivalent 
operation  in  the  spatial  domain  by  convolving  the 
image  with  the  impjlse  response  of  the  low-pass 
filter.  A  rigorous  implementation  of  this  con¬ 
volution  requires  the  integration  of  weighted 
intensity  over  the  entire  image  for  each  pixel. 
Either  of  these  techniques  requires  extremely 
involved  mathematics  to  process  complex  surface 
boundaries.  In  addition,  the  immense  computation 
load  required  is  prohibitive. 

Because  of  these  problems,  current  CIG 
syst  ems  have  retreated  from  proper  image 
prefiltering  to  a  variety  of  compromises  which 
fail  to  reduce  aliasing  to  an  acceptable  level. 

At  the  same  time  these  compromises  have  added  an 
undesirable  amount  of  computation  to  achieve  even 
a  limited  amount  of  antialiasing.  In  general, 
these  compromise  solutions  attempt  to  adjust  the 
intensity  of  any  pixel  which  contains  portions  of 
more  than  one  surface.  The  intensity  of  the 
pixel  is  represented  by  an  average  of  the 
intensities  of  all  surfaces  within  the  pixel, 
each  weighted  by  the  fraction  of  the  pixel  area 
that  it  covers.  Although  seemingly  straight¬ 
forward,  this  approach  is  very  costly  in  compu¬ 
tation  since  the  determination  of  subpixel  areas 
is  time-consuming  even  for  straight  edge  bound¬ 
aries.  To  simplify  the  computation  some  systems 
use  an  approximation,  subdividing  the  pixel  into 
an  nxn  array  of  subpixels  for  trfiich  the  quantized 
image  is  generated  and  averaged  for  the  total 
pixel.  The  problems  of  this  technique  are 
readily  apparent.  First,  the  computation  load  for 
an  antialiased  pixel  is  increased  by  a  factor  of 
n‘  (with  n  as  high  as  8).  Second,  a  quanti¬ 
zation  problem  still  exists  with  the  potential 
for  producing  distracting  jumps  in  Intensity  in  a 
dynamic  image.  Finally,  even  an  exact,  contin¬ 
uous  area  weighting  does  not  satisfy  the  sampling 
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theorem  filtering  requirement,  so  that  aliasing 
can  not  be  properly  suppressed.  In  brief,  cur¬ 
rent  techniques  provide  too  little  antialiasing 
at  too  great  a  cost. 

EFFECTIVE  ANTIALIASING 
The  Ideal  Solution 

The  theoretically  correct  solution  to  the 
aliasing  problem  is  to  filter  the  image  before 
sampling.  The  frequency  domain  transfer  function 
of  the  ideal  presampling  filter,  is 

shown  in  Fig.  1  (the  sample  spacing  is  assumed  to 
be  1  in  the  x  and  y  directions).  The  impulse 
response  of  this  filter  is: 

h(X(y)  s  .»  <  x,y  <  »  0) 


and  the  filtered  image  Fp(x,y)  is  this  con¬ 
volved  with  the  original  image  F(x,y): 


Fp(x,y)  *  !  !  h(x-R,y-S)F(R,S)dRdS  (2) 


Direct  evaluation  of  this  integral  for  each 
sample  >.s  computational  ly  very  costly.  It  is 
des’.able  to  find  some  approximations  to  this 
integral  which  give  suitable  results  without  a 
heavy  computational  burden. 

Approximating  the  Ideal  Solution 

Our  goal  is  to  develop  a  satisfactory 
approximation  to  the  convolution  in  Eq  (2)  while 
minimizing  the  computation  required.  There  are 
two  important  problems  that  we  must  solve. 

First,  we  must  find  a  way  to  perform  the 
convolution  before  sampling.  Second,  we  must 
drastically  reduce  the  number  and  complexity  of 
the  calculations. 

For  an  image  represented  by  a  set  of  sur¬ 
faces  and  their  boundaries  high  spatial  frequen¬ 
cies  will  occur  only  at  abrupt  changes  in  inten¬ 
sity.  Such  changes  occur  at  the  surface  bound¬ 
aries:  regions  within  the  boundaries  will  have 
low  spatial  frequencies  and,  therefore,  will 
require  no  antial iasing.  This  implies  that  anti¬ 
aliasing  can  be  limited  to  image  regions  near 
boundaries.  Thus  we  can  significantly  reduce  the 
computation  of  Eq  (2)  by  restricting  it  to  only  a 
fraction  of  the  image  points. 

We  must  further  limit  the  computation  by 
placing  finite  limits  on  the  integral  in  Eq  (2). 
We  will  do  this  by  approximating  the  impulse 
response,  Eq  (I),  by  a  weighting  function  with  a 
finite  domain.  This  domain,  which  we  will  call  a 
sampling  region,  must  cover  a  distance  of  at 
least  two  sampling  periods  in  x  and  y  for  accep¬ 
table  filtering.  Restricting  the  calculation  of 
the  convolution  not  only  reduces  the  computation 
load  dramatically,  but  also  allows  us  to  make  two 
more  simplifications  within  each  sampling  region. 
First,  within  each  sampling  region,  we  will 
approximate  all  boundaries  as  straight  lines. 
Second,  we  will  assume  surface  intensity  to  be 
constant  within  the  sampling  region.  With  these 
approximations  Eq  (2)  reduces  to 


Fc(x,y)  *  l  //  h'(x,y)dxdy  (3) 

F  1=1  1  Rt 


for  N  surfaces  visible  within  the  sampling  region 
R,  with  the  itf*  surface  visible  over  a  region 
N 

R-j  so  that  £  R,-  =  R.  The  function  h'(x,y) 

i=l 

is  the  weighting  function  approximation  to 
h(x,y). 

For  two  surfaces  separated  by  a  single 
boundary,  Eq  (3)  becomes 


F,(x,y)  =  I,  //  h' (x.yjdxdy+l-  //  h'(x,y)dxdy 
F  *1  R-Rl  (4) 

We  have  now  reduced  the  convolution  to  a  form  in 
which  the  integration  is  dependent  only  on  the 
region  of  integration.  In  Eq  (4)  the  regions  of 
integration  are  determined  by  the  location  of  the 
boundary.  We  have  developed  a  technique  to  eval¬ 
uate  the  integral  of  our  weighting  function 
explicitly  as  a  function  of  a  single  continuous 
measure  »  L.of  boundary  position  within  the 
sampling  region.  This  enables  us  to  express  the 
contribution  of  surface  1  to  the  filtered  inten¬ 
sity  of  a  sampling  region  as 

Ff  (x,y)  =  I.W (l)  (5) 

F1  1 

where  W( i)  is  a  new  weighting  function  applied  to 
the  surface  intensity  and  defined  by 


W(t)  =  ff  h‘ (x.y)dxdy  (6) 


W(t)  is  thus  the  volume  defined  under  h‘(x,y) 
over  R£  and  can  be  normalized  to  a  maximum  of 
one.  W(t)  will  equal  zero  when  the  surface  lies 
outside  the  sampling  region,  will  equal  one  when 
the  surface  covers  the  region  and  will  vary 
continuously  and  nonlinearly  as  the  boundary 
moves  across  the  region.  Since  the  total  volume 
under  h’(x»y)  over  R  will  be  normalized  to  one, 

Eq  (4)  can  be  written  as 

Fp(x,y)  =  I,WU)  +  I2(l-W(i))  (7) 

Equation  (7)  represents  a  manageable  approx¬ 
imation  of  the  ideal  filtering  defined  by  Eq  (2). 
We  have  achieved  a  tremendous  reduction  in  compu¬ 
tation  by  restricting  filtering  to  image  points 
near  boundaries,  approximating  the  impulse  re¬ 
sponse  by  a  bounded  weighting  function,  linear¬ 
izing  the  boundary  and  assuming  constant  inten¬ 
sity  with in  a  sampling  region,  and  devising  a 
means  of  representing  surface  intensity  weighting 
as  a  function  of  boundary  position  in  the  sam¬ 
pling  region.  In  addition  to  its  computational 
efficiency,  this  technique  has  the  Important 
feature  of  approximating  the  Ideal  filtering  on 
the  unsampled  image.  Because  the  filtering 
operation  depends  on  a  single  continuous  measure 
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of  boundary  position,  it  will  not  introduce  any 
quantization  effect  that  could  produce 
distracting  jumps  in  intensity  in  a  dynamic 
image. 

Demonstration  of  Results 

We  demonstrated  the  effectiveness  of  our 
antialiasing  technique  on  extreme  cases  vrtiere 
aliasing  is  severe  --  thin  bright  surfaces  on  a 
dark  background.  We  defined  surfaces  with  width 
equal  to  one  pixel  spacing  using  both  linear  and 
curved  boundaries.  The  surface  intensity  was 
assigned  the  maximum  value  (1)  and  the  background 
was  assigned  the  minimum  value  (zero).  Because 
each  surface  was  so  thin,  in  general  two  bound¬ 
aries  passed  through  a  given  sampling  region. 

Accordingly,  we  modified  Eq  (7)  to  represent  the 
contribution  of  the  surface  between  the 
boundaries 

Fp(x,y)  =  !,(«(*,)  +  W(t2)-1)  +  I2(2-W(*.1  )-W(e.2)) 
=  W(t1 )  +  wu2)-i 
since  1^=1  and  I2  =  0. 

To  provide  a  benchmark,  we  used  a  current 
technique  of  area-weighted  intensity  as  approx¬ 
imated  by  the  n  x  n  subpixel  array.  Since  no 
improvement  in  image  quality  was  perceivable  for 
n  greater  than  8,  we  used  this  value  as  represen¬ 
tative  of  exact  area-weighting. 

To  demonstrate  aliasing  clearly  we  generated 
a  128-pixel-wide  image  of  three  thin  surfaces 
with  linear  bounds  at  small  angles  to  the  scan 
lines.  Figure  2  shows  the  unfiltered  image  with 
and  without  zoom-central  enlargement  by  our  scan 
converter.  Antialiasing  using  the  n  x  n  subpixel 
array  technique  for  values  of  n  ranging  from  2  to 
8  is  shown  in  Fig.  3a,  b  and  c.  Although  consid¬ 
erable  smoothing  is  apparent,  some  aliasing  is 
still  evident  in  the  variation  in  intensity  in 
the  two  lower  lines.  This  residual  aliasing 
holds  the  potential  for  distracting  edge  crawling 
in  a  dynamic  display. 

Figure  3d  shows  the  same  image  antialiased 
by  our  technique.  Since  no  variation  in  inten¬ 
sity  is  perceivable  along  any  of  the  lines,  and 
since  filtered  intensity  at  any  pixel  is  a 
continuous  function  of  boundary  position,  no 
dynamic  discontinuities  will  be  produced  as  the 
boundary  moves  in  the  image. 

To  demonstrate  the  behavior  of  our  technique 
on  surfaces  with  curved  boundaries,  we  generated 
a  test  image  of  seven  elliptical  surfaces  of  1 
pixel  width.  Figure  4  shows  the  unfiltered  image 
with  and  without  enlargement.  Figure  5a,  b  and  c 
shows  the  image  smoothed  by  the  nxn  subpixel 
technique  for  values  of  n  ranging  from  2  to  8. 
Evidence  of  aliasing  is  still  present.  Figure  5d 
shows  the  image  after  filtering  with  our  tech¬ 
nique.  All  perceivable  aliasing  has  been 
removed. 

CONCLUSION 

Despite  the  assumptions  and  approximat ions 
we  used  to  make  our  technique  computationally 
efficient,  it  performs  antialiasing  in  a  manner 


far  superior  to  the  area-weighted  intensity  tech¬ 
nique  because  our  technique  represents  a  much 
better  approximation  to  ideal  filtering.  First, 
it  uses  nonuniform  weighting  of  intensity,  and 
second,  it  applies  the  weighting  over  a  larger 
region.  To  demonstrate  the  importance  of  each  of 
these  factors,  we  generated  a  test  image  of  21 
concentric  circles  each  1  pixel  wide.  Figure  6a 
shows  the  unfiltered  image.  When  uniform 
weighting  is  applied  over  two  sampling  periods 
(Fig.  6b),  the  region  is  large  enough  but  the 
weighting  is  improper.  Figure  6c  shows  the 
result  of  applying  nonuniform  weighting  over  a 
region  covering  only  one  sampling  period.  Figure 
6d  shows  the  result  of  applying  uniform  weighting 
over  one  sampling  period,  equivalent  to 
area-weighted  intensity  in  which  both  weighting 
and  region  size  are  improper.  Effective 
antialiasing  using  nonuniform  weighting  over  two 
sampling  periods  is  shown  in  Fig.  6e. 

The  success  of  our  technique  is  due  to  the 
fact  that  it  is  based  on  proper  filtering  before 
sampling.  Because  it  uses  a  continuous  measure 
of  boundary  position  it  eliminates  the  potential 
for  distracting  jumps  in  intensity  in  a  dynamic 
image.  The  technique  can  be  extended  to 
antialias  surfaces  with  multiple  boundaries,  such 
as  small  objects,  by  combining  the  position 
information  of  each  boundary.  Since  each  posi¬ 
tion  is  measured  continuously,  the  combined 
weighting  of  surface  intensity  will  vary  without 
discontinuity. 

We  have  demonstrated  our  technique  on  images 
including  boundaries  composed  of  lines  or 
second-order  curves.  However,  curves  of  any 
order  can  be  handled  as  long  as  the  equation  of 
the  boundary  is  known  analytically. 

Finally,  the  effectiveness  of  our  technique 
is  achieved  at  a  low  computational  cost.  The 
intensity  weighting  function  depends  on  a  single 
measurement  of  boundary  position  and  involves  a 
simple  computation  or  table  lookup.  In  the  test 
images  generated  our  technique  took  less  compu¬ 
tation  time  than  the  nxn  subpixel  array  tech¬ 
nique  with  n  =  2. 
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(a)  Normal  Size  Image  on  512  Pixel  Wide  Display  (b)  Same  Image  Enlarged  to  Show  Aliasing  More 

Clearly 

Fig.  2  Unfiltered  Image  of  Thin  Surfaces  with  Linear  Boundaries 


(c)  8  x  8  Subpixel  Array  (d)  Nonuniform  Weighting  Over  Two  Sampling  Periods 

Fig.  3  Antialiasing  of  Linear  Boundary  Image 
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(a)  Normal  Size  Image  on  a  512  Pixel  Wide  Display 


(b)  Same  Image  Enlarged  to  Show  Aliasing  More 
Clearly 

Fig.  4  Unfiltered  Image  of  Thin  Surfaces  with  Curved  Boundaries 


(a)  2  x  2  Subpixel  Array 


(b)  4  x  4  Subpixel  Array 


(c)  8  x  8  Subpixel  Array 


(d)  Nonuniform  Weighting  Over  Two  Sampling  Periods 


Fig.  5  Antialiasing  of  Curved  Boundary  Image 


(a)  Unfiltered  Image 


(b)  Uniform  Weighting  Over  Two  Sampling  Periods 


(e)  Nonuni  form  Weighting  Over  Two  Sampling  Periods 
Fig.  6  Effect  of  Weighting  Distribution  and  Region  on  Antialiasing  of  21  Concentric  Circles 
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ABSTRACT 


Wide  angle  visual  display  systems  used  in  flight  simulation  systems  often  exhibit 
distortion  due  to  the  display  geometry  and/or  optics.  Distortion  correction  in  the 
image  generator  permits  the  use  of  a  wide  variety  of  display  systems  with  the  assurance 
that  the  correct  perspective  of  the  environment  is  always  presented  to  the  viewer.  The 
paper  will  address  the  definitions  of  geometric  and  optical  distortion  followed  by  a  dis¬ 
cussion  of  some  parameters  relating  to  Computer  Image  Generation  (CIG)  and  display  system 
performance.  Several  kinds  of  displays  including  domes,  on-or-off  axis  infinity  systems, 
will  be  considered,  together  with  the  image  projection  devices  used  in  these  displays, 
including  unshapeable  light  valve  projectors  with  f-tan  6,  f-t? ,  or  anamorphic  lenses,  laser 
displays,  or  dynamically  changing  projectors.  Some  discussion  of  the  distortion  correction 
implemented  on  the  Aviation  Wide  Angle  Visual  System  (AWAVS)  and  Visual  Technology  Research 
Simulator's  (VTRS)  two  flight  simulators  CTOL  and  VTOL  at  the  Naval  Training  Equipment  Center 
(NTEC).  Orlando,  Florida  will  be  presented. 


INTRODUCTION 

An  inherent  problem  that  confronts  wide  angle 
visual  systems  is  distortion.  The  dominant  con¬ 
tributors  to  distortion  in  systems  of  this  type 
are  the  image  generator  and  the  display  system. 
Some  examples  would  be  a  Camera  Model  Image  Gen¬ 
erator,  a  Dome  Display  System,  and  an  Off-Axis  In¬ 
finity  Display.  In  the  past,  distortion  compensa¬ 
tion  for  wide  angle  systems  has  been  by  camera 
raster  shaping,  display  raster  shaping,  and  by 
purely  optical  means  0),  U),  (3).  Until  now. 
Wide  Angle  Visual  Systems  employing  Computer  Image 
Generators  have  had  to  rely  on  the  display  to 
totally  compensate  for  the  distortion.  This  has 
limited  the  use  of  available  display  devices  not 
before  considered  viable  for  wide  angle  applica¬ 
tions.  Thus  the  advent  of  CIG  distortion  correc¬ 
tion  will  make  these  and  future  display  technol¬ 
ogies  available. 

OBJECTIVES 

It  is  apparent  that  wide  field  of  view,  visual 
simulation  is  one  of  simulation's  most  cri¬ 
tical  deficiencies  Since  Computer  Image  Gen¬ 

eration  has  established  itself  as  the  prime  source 
for  wide  field  of  view  systems,  the  display  is  now 
the  major  obstacle  facing  systems  of  this  type. 
Distortion  and  methods  of  correction  have  com¬ 
pounded  this  problem  by  limiting  the  number  of 
display  types  and  configurations  that  can  be  used 
with  a  CIG  system.  This  being  the  case,  one  could 
conclude  that  the  CIG  must  play  a  more  integral 
role  in  the  total  visual  system;  that  is,  take  ad¬ 
vantage  of  its  flexibility  to  eliminate  the  dis¬ 
play  system's  distortion. 

The  Visual  Training  Research  Simulator  (VTRS) 
at  NTEC  made  one  of  its  primary  goals  the  investi¬ 


gation  and  implementation  of  CIG  comprehensive  dis¬ 
tortion  correction.  This  correction  was  to  be  ap¬ 
plied  to  those  distortions  encountered  in  dome  type 
displays.  The  objectives  were  to  apply  this  correc 
tion  to  a  variety  of  display  types  and  configura¬ 
tions.  This  included  light  valves  with  anamorphic 
f-J  lenses,  symmetric  f-t?  lenses,  f-tan  t?  lenses 
and  the  geometric  distortions  that  resulted  from  a 
particular  display  configuration.  Additional 
objectives  were  to  demonstrate  channel  matching 
between  two  light  valve  projectors,  with  f-f  lenses 
and  demonstrate  real-time  dynamic  correction.  This 
would  be  required  for  a  variable  area  of  interest 
inset. 

The  CIG  comprehensive  distortion  correction  im¬ 
plemented  was  required  to  be  flexible  enough  to 
compensate  for  any  type  or  form  of  distortion. 


Distortion  in  Dome  Displays 

The  dome  display  offers  versatility  in  the  me¬ 
chanical  layout  of  a  flight  simulator  and  will 
often  aid  the  design  concept  in  the  areas  of  safety 
and  reliability  while  yielding  a  wide  field  of  view 
with  moderate- to-good  resolution,  with  a  minimum 
of  display  channels.  Dome  displays,  however, 
suffer  from  distortion  arising  from  two  causes. 
Geometric  distortion  is  a  result  of  the  viewpoint 
and  projection  point  not  coinciding;  therefore, 
images  of  straight  lines  from  the  projector  are 
seen  to  be  curved  at  the  viewpoint.  Another  source 
of  distortion  is  the  projector  optics.  In  a  sys¬ 
tem  with  CIG  input,  the  view  window  (and  raster 
definition)  i,  f-tan  t*,  and  hence  even  a  pure  f-t? 
lens  would  yield  distortion  of  the  image.  The 
combined  result  of  these  two  distortions  is  asym¬ 
metrical,  and  is  not  readily  corrected  by  simple 
optical  methods. 
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Types  of  Distortion 

Geometric  distortion  in  dome  systems  arises 
from  two  causes:  dome  curvature  and  the  displace¬ 
ment  of  the  projection  system  from  the  viewpoint. 
Interestingly  enough,  the  first  would  not  cause 
distortion  if  the  latter  were  not  the  case.  Since 
the  projector  lens  cannot  be  placed  at  the  eye- 
point,  the  image  of  a  straight  line  from  the  pro¬ 
jection  optics  on  the  dome  will  look  curved  when 
seen  from  the  viewpoint.  There  will  also  be  a 
keystone  effect  similar  to  that  given  by  an  over¬ 
head  projector  whose  optical  axis  is  tilted.  Fig¬ 
ure  1  shows  schematically  the  problem  of  geometric 
distortion. 


Figure  1.  Geometric  Distortion 

Optical  distortion  is  characterized  by  depar¬ 
ture  of  a  lens  mapping  from  f  tan  9  behavior.  In 
this  discussion  we  shall  assume  that  the  depth  of 
focus  is  infinite,  i.e.,  the  lens  pupil  is  infini¬ 
tesimal.  An  f  tan  9  lens  maps  according  to  the 
general  rule  that  the  image  height  from  the 
optical  axis  to  the  image  point,  is  proportional 
to  the  object  height  (i.e.,  the  distance  from  the 
optical  axis  to  the  object  point).  A  schematic 
representation  is  given  in  figure  Z.  A  system 
with  no  distortion  results  in  the  following 
equality. 


Pa*  O'/PiO  =  PJe0'/Pe0. 

A  system  departing  from  this  expression  yields  dis 
tortion  to  the  extent  that  P'E  departs  from  P'jE  . 


Figure  2.  F-Tan-9  Lens  with  Distortion 


Wide  angle  systems,  such  as  that  at  VTRS,  often  use 
a  lens  with  an  f-9  mapping;  that  is,  the  angle  from 
the  optical  axis  to  the  image  point  is  proportional 
to  the  object  height.  This  type  of  lens  is  prac¬ 
tical  for  such  systems  since  it  equalizes  the  angu¬ 
lar  resolution  to  the  eye  over  the  wide  field.  How¬ 
ever,  it  is  not  compatible  with  CIG  systems  as  they 
are  currently  defined,  since  they  require  a  flat 
rectangular  view  window  for  edge  definition.  (A 
non-planar  view  window  introduces  the  difficulty 
that  edges  in  the  environment  project  to  a  curve  in 
the  view  window  that  is  not  a  linear  function  of 
scan  line  and  element  number.)  The  projection  of 
an  f9  lens  image  onto  a  flat  surface,  shown  in 
figure  3  has  pincushion  distortion. 


Figure  3.  Flat  Plane  Projection  of  fe  and  f  Tan 
0  Lenses;  Where  9'  <  9 

When  we  consider  the  view  window  definition  and 
the  way  that  a  CIG  system  processes  image  data,  we 
find  that  a  wide  angle  f-9  lens  system  results  in 
an  artificial  compression  of  the  image  near  the 
center  of  the  field  of  view  and  an  artificial  ex¬ 
pansion  of  the  image  near  the  edge.  The  key  to 
this  effect  can  be  seen  by  considering  an  example. 

The  general  equation  for  determining  the  angle 
8  to  a  scan  line  L,  if  Lm  is  the  number  of  lines 
from  the  optical  axis  to  the  edge  of  the  field  0m 
is  the  angle  to  the  edge,  is  given  by  6  =  tan'1 
[L(tan  e_)/Lm],  As  an  example,  suppose  the  half¬ 
field  (tne  angle  from  the  optical  axis  to  the  edge 
of  the  field)  is  40°  and  over  this  half-field  we 
have  400  raster  lines.  Suppose  a  face  segment  ex¬ 
tends  from  line  300  to  line  400  in  the  CIG  view 
window.  Then  the  angle  that  should  be  subtended 
by  it  is  given  by; 


,,„-l  ,400  tan  40°, 
tan  ( - m - ) 


tan 


-1 


,300 

400 


tan  40°) 


=  40°  -  32.183°  =  7.187° 


However,  if  lines  300  to  400  are  displayed 
through  a  projector  with  an  fe  lens,  lines  subtend 
an  angle  of  10°.  Hence  the  displayed  image  would 
be  larger  than  it  should  be,  i.e.,  there  is  an 
artificial  magnification  near  the  edge  of  the 
field.  Now  suppose  a  face  segment  extends  from 
line  0  to  line  100.  The  angle  it  should  subtend  is 
equal  to  tan'1  (1/4  tan  40°)  *  11.847°,  but  again 

it  would  subtend  an  angle  of  only  10°  when  pro¬ 
jected  through  the  f-9  lens.  Hence  there  is  an 
artificial  compression  near  the  center  of  the  field. 
Keep  in  mind  that  this  image  compression  refers 
to  the  scene  content  and  not  to  the  display 
resolution. 
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The  distortion  seen  on  wide  angle  displays  is  a 
combination  of  the  types  of  distortion  given  above, 
and  is  not  symmetric,  due  to  geometric  distortion. 
In  the  past,  simple  optical  methods  have  not  been 
successful  to  correct  this  kind  of  distortion. 

CIG  Distortion  Correction 


The  General  Approach  to  Distortion  Correction 

The  real-time  distortion  correction  hard¬ 
ware  for  an  edge-oriented  CIG  system  essen¬ 
tially  resides  between  Frames  II  and  III  of 
the  system;  where  Frame  II  processes  object 
and  face  data  and  generates  edges,  and  Frame 
III  determines  the  video  entries  for  line-by¬ 
line  raster  scan.l^)  The  basic  function  of 

the  CIG  distortion  correction  is  to  map  ver¬ 
tices  so  that  they  will  project  to  the  right 
places  relative  to  the  viewer,  and  to  compen¬ 
sate  for  curvature  introduced  by  the  system. 
Essentially  the  geometric  and  optical  distor¬ 
tion  is  a  view  window  problem  and  needs  to  be 
applied  to  the  system  for  the  purpose  of  de¬ 
fining  new  edges  immediately  after  the  edge 
definition  is  made.  Exceptions  to  this  char¬ 
acteristic  are  those  aspects  of  CIG  that  are 
handled  by  incrementers  in  Frame  III,  namely 
texturing  and  horizon  fading.  To  implement 
these  features  properly,  one  must  perform  a 
pixel-by-pixel  mapping  in  Frame  III  while 
these  functions  are  being  performed. 

Mapping  of  Edges 

The  edge  mapping  reguired  for  distortion 
correction  takes  edges  from  the  viewer  raster 
and  maps  them  to  the  projector  raster.  This 
involves  a  mapping  from  the  viewer  to  the  dome 
surface,  followed  by  a  mapping  from  the  dome 
surface  to  the  exit  pupil  of  the  projection 
lens,  and  followed  in  turn  by  a  mapping  to 
the  projector  raster.  Two  possible  outputs  to 
this  mapping  have  been  considered.  One  is  to 
define  the  image  of  the  edge  as  a  second-order 
function  of  scan  line  and  element  number,  that 
is  a 1 1 ?  +  ajJ-’  +  a 3 1 J  +  a* I  +  a5J  + 
a&  =  0. 

This  would  require  the  solution  of  six  equa¬ 
tions  and  six  unknowns  for  each  edge,  in  addi¬ 
tion  to  mapping  four  additional  points  per 
edge.  The  other  approach  would  be  to  segment 
each  Frame  II  edge  into  enough  pieces  to  ensure 
that  the  resulting  image  is  linear  to  the 
viewer.  This  would  require  an  increase  in  the 
edge  >  tpacity  of  Frame  III  and  would  also  in¬ 
crease  the  edge  burden  per  scan  line  approxi¬ 
mately  10  percent.  For  an  edge  oriented  CIG 
system 

hardware  required  is  less  for  the  second  (i.e., 
segmentation)  approach,  and  hence  that  was 
chosen. 

Segmentation  of  Edges 

The  goal  of  the  edge  breakup  is  to  segment 
the  edge  into  enough  pieces  so  that  each 
piece,  when  mapped  by  the  distortion  correc¬ 
tion  map  looks  like  a  straight  line  on  the 
projector  raster  to  within  a  maximum  allowed 
error  E_,  .  The  departure  from  linearity 
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of  the  mapping  of  one  of  these  segments  is  as¬ 
sumed  to  be  a  maximum  at  the  midpoint  of  tht 
edge.  Hence  we  talk  about  the  midpoint  error 
E  associated  with  representing  this  mapping  as 
a  segment.  This  is  illustrated  in  figure  4. 
Here  E  is  the  distance  from  the  mapping  of  the 
midpoint  of  the  viewer's  edge  to  the  straight 
line  segment  shown  in  the  projector  raster.  The 
maximum  allowed  midpoint  error  Emax  is  decided 
by  the  designer,  and  off-line  software  deter¬ 
mines  how  much  an  edge  needs  to  be  segmented  to 
get  E  to  within  Emax.  The  segmentation  factor 
depends  on  the  edge  position,  its  length,  and 
its  orientation. 


Figure  4.  Definition  of  Midpoint  Error 

Once  an  edge  is  segmented,  the  output 
edges  are  mapped  to  the  projector  raster  posi¬ 
tions  and  are  sent  to  Frame  III. 

A  fundamental  question  concerning  the  im¬ 
pact  of  segmentation  on  a  CGI  system  is  the 
number  of  segments  to  be  expected  from  a 
"typical"  scene.  A  scene  would  usually  con¬ 
sist  mostly  of  short  edges  that  need  only  be 
mapped,  but  not  segmented,  while  some  edges 
may  need  both.  An  edge  data  dump  was  taken 
from  the  CGI  Frame  II  for  several  scenes,  and 
this  data  was  mapped  and  segmented  sufficiently 
to  yield  a  given  "midpoint  error"  accuracy  for 
the  VTRS  Conventional  Carrier  Take-Off  and  Land¬ 
ing  Simulator,  (CTOL)  background  and  target  lens 
systems.  Plots  of  the  fractional  increase  in 
edge  burden  (edges  out/edges  in)  versus  the 
midpoint  error  accuracy  in  number  of  pixels 
required  are  shown  for  the  background  lens  in 
figure  5  and  for  the  target  lens  in  figure  6 
for  a  pair  of  scenes  (one  scan  with  the  USS 
Forrestal  taking  up  half  the  field  of  view  and 
one  scene  of  the  Meridian  Mississippi  airfield). 
Especially  for  the  background  lens,  there  is  a 
quite  significant  increase  in  edges  due  to 
segmentation,  even  if  the  mapping  accuracy  is 
only  one  pixel. 

RESULTS 

Distortion  correction  has  been  applied  to  the 
VTRS  CGI  and  display  systems.  In  the  VTOL  system, 
the  distortion  correction  has  been  simultaneously 
applied  (real  time)  to  two  identical  background 
channels,  each  with  its  own  view  window  orientation, 
and  with  its  90°  horizontal  by  70°  vertical  fields 
of  view  (114°  diagonal  fields)  using  GE  color  light 
valve  projectors  and  radially  symme'ric  f-6  lenses. 
The  layout  of  the  dome.  Lamps  Mark  III  helicopter 
mockup  cockpit,  and  optics  is  shown  schematically  in 
figures  7  and  8.  The  projector  and  optics  is  shown 
in  figure  9.  Of  particular  importance  in  this  lay¬ 
out  is  the  downward  field  of  view  critical  for  car¬ 
rier  deck  landing  training. 
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Figure  7.  Elevation  View  of  VTOL  Cockpit 
Configuration 


This  mechanical  configuration  allows  tremendous 
flexibility  in  regard  to  projection  attitude.  The 
lens  can  be  rotated  360°  in  two  axis;  one  of  them 
is  about  the  original  optical  axis  of  the  light 
valve,  and  the  other  is  the  rotation  of  the  projec¬ 
tion  head  about  the  axis  defined  by  the  long  leg  of 
the  lens  assembly.  A  pechan  prism  is  provided  to 
compensate  for  any  image  roll  introduced.  This 
provides  the  ability  to  juxtapose  many  projectors 
resulting  in  a  large  continuous  field  of  view. 

In  the  CTOL  system  the  distortion  correction 
has  been  applied  statically  to  a  160°  x  80°  back¬ 
ground  field  of  view,  using  a  monochrome  GE  light 
valve  projector  and  a  wide  angle  f-9  anamorphic 
lens.  A  schematic  layout  of  the  optics,  dome,  and 
the  T2C  cockpit  is  shown  in  figure  10. 


field  prior  to  distortion  correction. 

CONCLUSIONS 

Thus  far  Comprehensive  Distortion  Correction 
(CDC)  has  been  successfully  applied  to  dome  dis¬ 
play  systems  with  schlieren  type  light  valves  and 
projection  lenses.  The  flexibility  of  CDC  has 
made  it  attractive  to  other  wide  angle  type  dis¬ 
play  systems;  for  example  laser  displays.  In  line 
Infinity  Optical  Systems  (ILIOS)  and  off  axis 
infinity  display  systems. 

Laser  displays  with  their  varying  scanning 
techniques  invariably  introduce  distortion.  Some 
laser  display  systems  require  nonconventional 
image  generator  inputs  that  are  required  to  com¬ 
pensate  for  the  laser  projector  distortion  (’). 
CDC  can  remedy  this  by  offering  the  display  de¬ 
signer  the  freedom  to  use  a  wide  variety  of 


The  real-time  correction  for  the  CTOL  is  now 
in  progress  for  both  the  background  and  the  target 
lens  (the  target  lens  is  a  moveable  f-tan  8  lens 
with  zoom).  The  CIG  transport  delay,  i.e.,  the 
time  between  a  Frame  I  Operational  Flight  Trainer 
input  and  the  display  of  that  frame,  was  increa¬ 
sed  by  a  field  during  implementation  of  distor¬ 
tion  correction,  due  to  the  necessity  of 
retrofitting  distortion  correction  to  an  existing 
system.  A  comprehensive  CIG  design  incorporating 
distortion  correction  should  eliminate  this 
delay. 

The  real-time  application  of  distortion  cor¬ 
rection  at  the  present  time  includes  edges,  curved 
surface  shading,  and  fading.  Study  efforts  are 
currently  defining  the  application  with  texturing, 
oversampling  smoothing,  and  dynamic  priority. 

Photographs  have  been  taken  of  displayed 
imagery  in  the  CTOL  system  with  and  without  dis¬ 
tortion  correction.  A  pair  of  such  photos  showing 
the  U.S.S.  Forrestal  is  given  in  figures  11  and 
12.  Inspection  of  these  photos  reveals  the 
curvature  introduced  by  distortion,  and  its  cor¬ 
rection.  Also  one  can  notice  the  artificial  mag¬ 
nification  of  the  carrier  near  the  edge  of  the 


scanning  concepts  that  best  suit  a  particular 
visual  system.  It  must  be  noted  that  the  level  of 
CDC  complexity  is  directly  coupled  to  the  scanning 
method.  For  example,  distortion  introduced  by  a 
typical  raster  scanning  technique  is  more  readily 
corrected  than  for  one  that  requires  an  annular 
image  input.  A  particular  scanning  method  could 
impact  the  image  generator  and  visual  system  in 
one  or  all  the  following  ways:  add  a  frame  time 
delay,  a  need  for  output  field  buffers,  the 
necessity  for  high  speed  digital  to  analog  con¬ 
verters,  or  the  number  of  displayed  edges.  These 
effects  can  be  reduced  or  eliminated  if  trade-offs 
are  exercised  early  in  the  display  and  image  gener¬ 
ator  design. 

CDC  can  offer  ILIOS  ^  a  feature  not  before 
achieved  with  an  image  generator  input,  regardless 
of  the  display  input  i.e.,  CRT's,  light  valves,  or 
lasers.  This  feature  is  to  provide,  to  the  ob¬ 
server,  raster  lines  that  subtend  equal  angles. 
Presently  the  Advanced  Simulator  for  Pilot  Train¬ 
ing  (ASPT)  visual  system  displays  the  raster  lines 
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axis  than  in  the  display  center  which  is  contrary 
to  the  way  displays  normally  perform.  CDC  can 
simplify  and  improve  the  display  design  and  perfor¬ 
mance  in  systems  of  this  type,  while  applying  the 
same  correction  algorithm  to  each  channel. 

CDC  also  can  be  applied  to  off  axis  infinity 
displays  that  enherently  have  asymmetric  distortion 
characteristics  that  are  normally  compensated  by 
optical  means. 

In  summary,  CDC  provides  the  visual  simula¬ 
tion  community,  with  a  necessary  ingredient  that 
will  aid  in  the  design  of  wide  angle  display  sys¬ 
tems.  Since  its  conception  three  years  ago,  it 
has  proven  itself  as  a  viable  addition  to  the  total 
visual  system.  With  its  feasibility  clearly  demon¬ 
strated  at  VTRS  present  and  future,  visual  systems 
can  take  advantage  of  CDC  in  their  display  design. 


Figure  11.  Scene  of  USS  Forrestal  Projected  on 
Dome  Exhibiting  Anamorphic  Distortion 


Figure  12.  Same  Scene  as  Figure  11  with  Distor¬ 
tion  Corrected 


as  the  tangent  of  the  angle.  On  ASPT,  this 
results  in  higher  angular  resolution  (approxi¬ 
mately  1.5X)  between  raster  lines  at  the  extreme 
field  (+45°)  than  in  the  center.  This  higher 
resolution  area  is  wasted  in  the  region  of  the 
total  field  of  view  of  the  optical  system.  This 
means  that  this  region  is  only  visible  with  head 
motion  and  not  from  the  nominal  eyepoint.  However, 
with  the  raster  distributed  at  equal  angles,  a 
uniform  resolution  is  obtained  throughout  the  IUDS 
field  of  view  and  across  channels.  This  also 
simplifies  the  display  in  that  it  relaxes  the 
requirement  for  a  higher  resolution  off 
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COMPUTER  GENERATION  OF  CURVILINEAR  OBJECTS 


Johnson  K.  Yan,  Staff  Engineer 
Singer  -  Link  Division 
Sunnyvale,  California  94086 


ABSTRACT 

This  paper  surveys  alternative  approaches  to  curvilinear  object  generation  by  computer. 
These  alternative  approaches  are  compared  in  terms  of  ease  of  generation  of  silhouettes  of 
objects,  ease  of  intensity  computation,  ease  of  texture  generation,  image  quality,  data  base 
compactness  and  flexibility  of  modeling  free-form  curvilinear  objects.  The  alternative 
approaches  surveyed  are  planar  surface  approach,  quadric  surface  approach  and  parametric  sur¬ 
face  approach.  Other  techniques  within  the  class  of  parametric  surfaces  which  appear  to  be 
promising  but  have  not  yet  been  tried  are  also  discussed. 

INTRODUCTION 


For  digital  presentation  in  support  of  flight 
simulation,  it  is  frequently  necessary  to  portray 
man-made  or  natural  curvilinear  objects  such  as 
water  towers,  oil  storage  tanks,  bomb  craters, 
silos,  aircrafts  and  rolling  terrain.  In  most 
existing  real-time,  computer-generated  image  (CGI) 
systems,  these  curvilinear  objects  are  approxi¬ 
mated  by  planar  polygons.  We  will  refer  to  this 
approach  as  the  planar  surface  approach. 

The  extensive  use  of  planar  surface  approach 
in  existing  real-time  CGI  systems  is  due  to  the 
ease  of  real-time  implementation  with  this 
approach.  However,  there  are  two  major  short¬ 
comings  of  this  simple  approach.  The  first  short¬ 
coming  of  planar  surface  approximation  for  curvi¬ 
linear  objects  is  that,  while  smooth  shading 
restores  smoothness  to  these  objects,  the  silhou¬ 
ettes  remain  angular.  The  second  shortcoming  is 
that  a  great  number  of  planar  surfaces  are  re¬ 
quired  for  good  approximations  of  curvilinear 
objects,  thus  inflating  the  size  of  the  data  base. 

It  is  these  shortcomings  of  planar  surface 
approach  that  have  stimulated  active  research  in 
search  of  alternative  approaches.  The  alternative 
approaches  known  are  mainly  results  of  studies  by 
universities  doing  research  in  computer  graphics 
and  by  the  training  equipment  industry.  These 
alternative  approaches  can  be  divided  into  two 
classes  of  approaches,  namely,  parametric  surface 
approach  and  quadric  surface  approach. 

In  the  following  sections  we  will  survey 
known  techniques  of  the  three  approaches  for 
curvilinear  object  generation,  namely,  planar 
surface  approach,  quadric  surface  approach  and 
parametric  surface  approach.  Techniques  within 
these  three  classes  of  approaches  will  be  compared 
in  terms  of  the  following  attributes: 

1.  Ease  of  generation  of  silhouettes  of 
objects. 


2.  Ease  of  Intensity  computation  within 
silhouettes  of  objects. 


3. 

Ease  of  texture  generation  within 

silhouettes  of  objects. 

4. 

Image  quality. 

5. 

Data  base  compactness. 

6.  Flexibility  of  modeling  free-form  curvi¬ 
linear  objects. 


COORDINATE  SYSTEMS 

For  the  coordinate  systems  used,  let  (xe,  ye, 
ze)  denote  coordinates  in  the  environment  (or 
earth)  coordinate  system;  let  (xp,  yp,  zp)  denote 
coordinates  in  the  pilot's  eye  coordinate  system. 
The  pilot's  eye  coordinate  system  is  such  that  the 
origin  is  at  the  pilot's  eye;  the  zp  axis  is  per¬ 
pendicular  to  the  image  plane.  The  pilot's  eye 
coordinate  system  and  the  "3-D  screen"  coordinate 
system  (xs,  ys,  zs)  are  related  by  a  perspective 
transformation: 


xs  "  xp/zp 
Vs  =  V^p 
Zs  =  l/zp 

This  perspective  transformation  has  the 
property  that  perspective  projection  (with  the 
pilot's  eye  as  the  vantage  point)  of  objects  onto 
the  image  plane  in  the  pilot's  eye  coordinate 
system  is  equivalent  to  orthographic  (vantage  point 
at  infinity)  projection  of  them  onto  the  image 
plane  in  the  "3-D  screen"  coordinate  system  (1). 
Therefore,  xs  and  ys  coordinates  in  the  "3-D 
screen"  coordinate  system  are  the  2-D  screen  coor¬ 
dinates  on  the  image  plane.  Notice  that  z$  is  the 
reciprocal  of  zp,  the  Z-depth.  Therefore,  zs  can 
be  used  directly  for  priority  algorithms  which 
employs  depth  comparisons. 

PARAMETRIC  SURFACE  APPROACH 

Parametric  surfaces  are  surfaces  defined  on 
two  independent  parameters  most  often  denoted  by  u 
and  v.  The  2-D  u-v  space  is  also  referred  to  as 
the  parametric  space.  The  two  parameters  are 


usually  normalized  to  lie  between  0  and  1.  Thus, 
a  normalized  u-v  space  is  a  unit  square.  In  the 
environment  coordinate  system,  a  general  para¬ 
metric  surface  is  defined  by  three  equations  in  u 
and  v  as  follows: 

xe  =  f(u,v)  [1] 

ye  *  g(u.v)  [2] 

ze  =  h(u,v)  [3] 


where  0<u ,  v<l . 
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The  above  equations  define  one  "patch"  of  an  object 
modeled  with  parametric  surfaces.  An  object  is 
usually  modeled  by  piecing  together  many  such 
patches.  Normally,  certain  orders  of  continuity 
(Cl  continuity  for  curvature  continuity,  C1  contin¬ 
uity  for  gradient  continuity,  and  C°  continuity  for 
position  continuity)  are  desired,  both  within 
patches  and  along  patch  boundaries,  to  give  the 
desired  degree  of  smoothness  to  the  appearance  of 
objects . 

The  complexity  of  the  three  functions,  f,  g 
and  h,  depends  on  the  order  of  continuity  desired. 
The  functions  f,  g  and  h  are,  in  general,  nonlinear 
in  u  and  v.  It  should  be  noted  that  when  f,  g  and 
h  are  linear  in  u  and  v,  the  patch  reduces  to  a 
planar  surface. 

In  the  following  subsections  we  will  discuss 
various  types  of  parametric  surfaces  and  the 
computations  involved  for  their  rendition. 

Bicubic  Patches 


Mathematical  Background.  Bicubic  patches  are 
those  parametric  surfaces  in  which  the  f,  g  and  h 
functions  in  equations  [1],  [2]  and  [3]  are  bicu¬ 
bic  in  u  and  v.  In  the  environment  coordinate 
system,  a  bicubic  patch  is  defined  as  follows: 

xe  =  UBXeBTVT  [4] 

ye  =  UBYeBTVT  [5] 

ze  =  UBZeBTVT  [6] 

where  U  =  (u3  u2  u  1)  and  V  =  (v3  v2  v  1)  are  row 

matrices;  the  superscript  T  indicates  matrix 
transposition.  Xe,  Ye  and  Ze  are  4x4  matrices 
defining  the  three  coordinates  of  16  "control 
points."  B  is  a  constant  4x4  matrix. 

For  the  same  surface,  the  type  of  control 
points  will  determine  the  constant  B  matrix  to  be 
used.  Coons'  control  points  consist  of  four  cor¬ 
ner  position  vectors  of  the  patch,  two  tangent 
vectors  (one  in  the  direction  u,  the  other  in  the 
direction  v)  and  one  "twist  vector"  in  the  four 
corners  (2).  This  information  usually  does  not 
provide  an  intuitive  “feel"  for  the  shape  of  the 
surface.  Therefore,  Coons'  patch  finds  little  use 
on  interactive  surface  design.  On  the  other  hand, 
control  points  of  Bezier  (3)  and  B-spline  (4) 
surfaces  consist  of  16  position  vectors.  Four  of 
these  16  position  vectors  are  the  four  corners  of 
the  patch  and  the  other  12  position  vectors,  even 
though  not  on  the  patch  itself,  have  the  effect  of 
attracting  (like  a  magnet  to  a  flexible  metal 
sheet)  the  patch  towards  them.  Bezier  and  B-spline 
patches  therefore  provide  the  intuitive  feel  for 
surface  design.  One  advantage  of  a  B-spline  sur¬ 
face  over  a  Bezier  surface  Is  that  the  effect  of 
moving  a  control  point  of  a  B-spline  surface  Is 
local  while  that  of  a  Bezier  surface  is  global. 


involved  in  rendering  a  true  perspective  view  of 
bicubic  patches  will  be  described  next.  The  two 
major  steps  in  rendering  a  perspective  view  of  an 
object  are  finding  the  object's  silhouette  and  the 
surface  normal  at  each  point  within  the  silhouette 
far  intensity  computation. 

Because  the  control  points  are  either  posi¬ 
tion  vectors  or  direction  vectors,  they  can  be 
spatially  transformed  to  the  pilot's  eye  coordi¬ 
nate  system  (rotation  for  direction  vectors  and 
translation  and  rotation  for  position  vectors). 

Let  the  surface  be  expressed  in  the  pilot's  eye 
coordinate  system  as  follows: 

xp  =  UBXpBTVT  =  UMVt  [7] 

yp  =  UBYpBTVT  =  UNVT  [8] 

zp  =  UBZpBTVT  =  ULVt  [9] 

where  M=BXpBT,  N=BYpBT,  and  L=BZpBT. 

Let  the  tangent  plane  to  the  surface  at  (u,v) 
be  given  by 

a(u,v)xp  +  b(u,v)yp  +  c(  d(u,v)=0  [10] 

Therefore  (a(u,v) ,b(u,v) ,c(u, . , /  is  the  surface 
normal  vector  (in  the  pilot's  eye  coordinates)  at 
(u,v).  Taking  equation  [10]  through  perspective 
transformation  we  have: 

a(u,v)xs  +  b(u,v)ys  +  d(u,v)zs  +  c(u,v)=0  [11] 

Therefore,  the  surface  normal  vector  (in  screen 
coordinates)  at  (u,v)  is  (a(u,v),b(u,v),d(u,v)). 

It  can  be  shown  that  a(u,v),  b(u,v),  c(u,v)  and 
d (u.v)  are  biquintic  in  u  and  v. 

The  surface  normal  vector  in  pilot's  eye 
coordinates  is  used  for  intensity  computation 
while  that  in  screen  coordinates  is  used  for  find¬ 
ing  the  silhouette  of  a  patch  discussed  below. 

Applying  perspective  transformation  to 


equations  [7],  [8]  and  [9],  we  have 

xs  =  UMVt/ULVt 

[12] 

ys  =  UNVt/ULVt 

[13] 

zs  =  1/ULVt 

[14] 

In  the  screen  coordinate  system  the  space 
curves  of  the  four  boundary  curves  of  the  patch 
are  given  by  substituting  u=0,  u=l,  v=0  and  v=l  in 
equations  [12],  [13]  and  [14].  These  four  bound¬ 
ary  curves  are  therefore  univariate  rational 
cubics  (rational  cubic  functions  of  one  indepen¬ 
dent  variable)  in  u  or  v.  The  plane  curves  of 
these  four  boundary  curves  on  the  screen  are  given 
by  the  x  and  y  components  of  the  corresponding 
space  curve  in  the  screen  coordinate  system. 


Regardless  of  whether  the  control  points  are 
expressed  in  Coons,  Bezier  or  B-spline  form,  they 
are  of  the  same  form  as  shown  in  equations  [4], 
[5]  and  [6].  It  can  be  shown  that  for  a  control 
point  matrix  in  one  form  there  exist  transforma¬ 
tions  to  compute  an  equivalent  control  point 
matrix  in  the  other  two  forms.  Rendering  of  bi¬ 
cubic  patches  will  be  discussed  ir.  terms  of 
equations  [4],  [5]  and  [6].  The  mathematics 


In  the  planar  surface  approach,  scan  line  in¬ 
tersections  of  a  boundary  edge  are  calculated  in¬ 
crementally.  However,  in  the  bicubic  patch  ap¬ 
proach,  in  order  to  get  the  intersection  of  a  par¬ 
ticular  boundary  curve  (say  v*0)  with  scan  line  k, 
we  first  have  to  solve  for  u  with  the  following 
cubic  equation  in  u: 
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[15] 


thus,  the  computations  are  much  more  expensive. 


u  =  UN  TO  0  0  11T  *  k 
UL  [0  0  0  1]T 

The  above  equation  can  be  rewritten  as: 

a]u3  +  b-]u2  +  C|U  +  dj 
32U^  +  bou^  +  C2U  +  d2 


To  obtain  (u,v)  for  a  picture  element  (k-|,  k2 ) 
on  the  patch  not  on  silhouette  or  boundary  curves, 
the  following  two  bicubic  equations  have  to  be 
solved: 

xs  *  UMVT/ULVT  =  lq  [19] 


Once  u  is  obtained,  x,,  a  rational  cubic  equation 
in  u,  can  be  evaluated. 

To  obtain  the  surface  normal  at  (u,0)  for 
intensity  computation,  (u,0)  is  substituted  into 
the  parametric  surface  normal  equations. 

The  four  boundary  curves  of  a  patch  do  not 
completely  define  the  silhouette  of  a  patch. 

Other  curves  (known  as  silhouette  curves),  which 
are  dynamically  defined  as  a  function  of  the  view¬ 
ing  direction,  are  necessary  to  completely  define 
the  silhouette  of  a  patch.  This  is  illustrated  in 
Figure  1. 


Figure  1 

Boundary  and  Silhouette  Curves  Constitute 
the  Silhouette  of  a  Patch 

The  silhouette  curve  of  a  surface  is  defined 
mathematically  as  the  locus  of  points  on  the  sur¬ 
face  which  have  a  zero  Z  component  of  the  normal 
vector  in  screen  coordinates  (i.e.,  d(u,v)=0).  It 
is  interesting  to  note  that,  from  equations  [10] 
and  [11],  requiring  d(u,v)*0  is  equivalent  to 
requiring  the  constant  term  of  the  tangent  plane 
in  pilot's  eye  coordinates  to  be  zero.  In  other 
words,  the  silhouette  edge  can  also  be  defined  as 
the  locus  of  points  on  the  surface  which  have  a 
tangent  plane  in  the  pilot's  eye  coordinate  system 
passing  through  the  eye  point. 

The  (u,v)  for  the  intersection  of  a  silhouette 
curve  with  scan  line  k  is  given  by  the  solution  of 
the  following  two  bivariate  polynomial  equations 
(polynomial  equations  of  two  Independent  vari¬ 
ables): 

ys  =  UNVt/UIVt  =  k  [17] 


d(u,v)  =  0  [18] 

The  first  equation  is  bicubic  in  u  and  v  while  the 
second  is  biqulntic  in  u  and  v.  For  scan  line 
Intersections  of  a  boundary  curve,  an  equation 
with  a  single  parameter  is  involved.  However,  for 
scan  line  intersections  of  a  silhouette  curve,  two 
equations  with  two  parameters  are  involved  and. 


ys  =  UNVt/ULVt  =  k2  [20] 

The  (u,v)  obtained  is  then  substituted  into  the 
surface  normal  equations  for  intensity  computation. 

In  the  following  sections  we  will  examine 
various  researchers'  approximations  to  the  rendi¬ 
tion  of  true  perspective  view  of  bicubic  patches 
in  order  to  simplify  the  computations. 

Catmull  Algorithm  (5).  Catmull  devised  the 
first  algorithm  to  render  curvilinear  objects 
modeled  with  bicubic  patches.  The  algorithm  is  as 
follows:  Each  bicubic  patch  is  recursively  subdi¬ 
vided*  into  smaller  subpatches  until  the  image  of 
each  subpatch  covers  only  one  picture  element. 

The  screen  coordinates  xs,  ys  and  zs  and  the  three 
components  of  the  surface  normal  in  pilot's  eye 
coordinates  a(u,v),  b(u,v)  and  c(u,v)  of  the  patch 
are  evaluated  and  carried  along  with  each  subdivi¬ 
sion.  We  will  refer  to  the  evaluation  of  a  function 
associated  with  a  patch  at  each  subdivision  of  the 
patch  as  a  subdivision  of  the  function.  When  the 
subpatch  covers  only  one  picture  element,  the 
surface  normal  is  used  to  compute  the  intensity  of 
the  subpatch  at  that  picture  element.  The  xs,  ys 
and  zs  screen  coordinates  are  used  to  perform 
hidden  surface  removal  employing  the  depth  buffer 
approach.  A  depth  buffer  is  a  frame  buffer  which 
provides  storage  for  intensity  and  depth  (zs) 
information  for  each  picture  element  of  the  screen. 
Initially,  the  intensity  information  of  the  whole 
frame  buffer  is  set  to  background  intensity  and 
the  depth  information  is  set  to  zero  (recall  that 
zs  is  the  reciprocal  depth).  When  a  subpatch 
covers  only  one  picture  element,  its  xs  and  y< 
screen  coordinates  are  used  to  address  the  buffer. 
The  zs  in  the  addressed  location  is  read  and  com¬ 
pared  with  the  zs  of  this  subpatch.  If  zs  of  this 
subpatch  is  larger  than  the  one  read,  then  the 
intensity  and  zs  of  this  subpatch  replaces  the  old 
values  in  the  addressed  location  of  the  buffer; 
otherwise,  nothing  is  done. 

Catmull  has  devised  an  efficient  algorithm  to 
subdivide  bivariate  polynomial  functions.  In 
particular,  subdivision  of  bicubic  functions  is 
quite  simple.  However,  as  seen  earlier,  the 
screen  coordinates  of  a  patch  are  rational  bicubic 
functions  which  do  not  fit  into  the  simple  sub¬ 
division  algorithm.  In  order  to  still  make  use 
of  the  simple  bicubic  subdivision  algorithm, 

Catmull  approximates  a  patch  in  screen  coordinates 
by  taking  the  control  points  through  the  perspec¬ 
tive  transformation  and  recreating  the  bicubic 
patch  in  screen  coordinate  systems  with  the  trans¬ 
formed  control  points.  He  claims  that  the  approx¬ 
imation  is  very  close. 


♦Subdivision  of  a  patch  is  the  process  of  sub' 
dividing  a  patch  into  four  smaller  subpatches 
along  the  midpoints  of  the  two  parameters. 
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As  mentioned  earlier,  tne  components  of  the 
surface  normal  of  a  bicubic  patch  are  biquintic. 

To  make  use  of  the  simple  bicubic  subdivision 
algorithm,  Catmull  approximates  the  surface  normal 
of  a  bicubic  patch  by  a  bicubic  function. 

For  texture  generation  on  bicubic  patches, 
Catmull  keeps  track  of  the  parametric  values  u  and 
v  as  each  subpatch  is  subdivided.  The  u  and  v 
parametric  values  are  then  used  to  look  up  a  table 
of  texture  modulation  values  defined  on  the  unit 
square  of  the  u-v  parametric  space.  This  texture 
modulation  value  is  then  added  to  the  intrinsic 
intensity  obtained  using  the  surface  normal. 

Blinn  Algorithm  (6) .  Despite  his  simple  sub¬ 
division  algorithm,  Catmull 's  algorithm  is  still 
quite  time-consuming  and  uses  a  large  amount  of 
memory.  To  overcome  these  problems,  scan  line 
algorithms  have  been  developed  for  bicubic  patches. 
Scan  line  algorithms  generate  the  picture  elements 
on  the  screen  from  left  to  right,  top  to  bottom, 
as  opposed  to  the  random  order  in  Catmull 's 
algorithm.  Scan  line  algorithms  allow  exploita¬ 
tion  of  coherence  of  images.  Blinn's  algorithm, 
as  well  as  algorithms  discussed  in  later  sections, 
are  scan  line  algorithms. 

In  scan  line  algorithms,  one  of  the  important 
tasks  is  to  find  the  silhouettes  of  the  patches. 
After  the  silhouettes  are  found,  tonal  computa¬ 
tions  (intensity  and/or  texture  modulation)  are 
performed  within  the  silhouettes  of  the  patches. 

For  boundary  curve  intersections  with  scan  lines, 
a  univariate  bicubic  function  of  equation  [16]  is 
involved.  Blinn  uses  Newton-Raphson  iteration 
technique  to  solve  for  the  independent  variable 
with  the  independent  variable  found  for  the  pre¬ 
vious  scan  line  as  the  initial  guess.  Blinn 
claims  that  convergence  is  quite  fast. 

For  silhouette  curve  intersections  with  scan 
lines,  a  pair  of  bivariate  polynomial  functions 
(one  bicubic  and  the  other  biquintic)  are  involved. 
Blinn  uses  the  bivariate  Newton-Raphson  iteration 
technique  with  the  two  parameters  found  for  the 
previous  scan  line  as  the  initial  guess.  Since 
bivariate  Newton-Raphson  iteration  technique  is 
much  more  involved  than  the  univariate  counterpart, 
generation  of  silhouette  curves  are  much  more 
difficult  than  boundary  curves.  After  the  indepen¬ 
dent  parametric  variables  u  and  v  are  found  for 
scan  line  intersections  of  the  boundary  curves  and 
the  silhouette  curves,  they  are  substituted  into 
equation  [12]  to  obtain  the  xs  coordinates  of  the 
intersections. 

For  intensity  computation  as  well  as  texture 
generation,  the  (u,v)  parametric  values  have  to  be 
obtained  for  each  picture  element  along  each  scan 
line  covered  by  a  patch.  The  (u,v)  parametric 
values  for  each  picture  element  on  each  scan  line 
have  to  be  obtained  by  solving  the  pair  of  bivar¬ 
iate  cubic  functions  of  equations  [19]  and  [20]. 

The  (u,v)  values  are  then  substituted  into  equa¬ 
tions  of  the  components  of  the  surface  normal  to 
obtain  the  surface  normal  for  intensity  computa¬ 
tion. 

To  avoid  having  to  compute  the  (u,v)  values 
for  each  picture  element  covered  by  a  patch, 

Blinn  introduced  a  generalized  silhouette  concept 
to  generate  (u,v)  for  the  scan  line  intersections 
of  16  generalized  silhouette  curves  on  a 


particular  scan  line  (say  k)  by  solving  the  fol¬ 
lowing  equations  for  16  different  0: 

ys  =  UNVt/ULVt  =  k 

a(u,v)sine  *  d(u,v)cos0  -  0 

Note  that  0=0°  and  180°  correspond  to  the  silhou¬ 
ette  curves  discussed  earlier.  The  intensity  is 
then  calculated  exactly  at  the  scan  line  inter¬ 
sections  of  these  16  generalized  curves.  In 
between  these  16  intersections  the  intensity  is 
linearly  interpolated. 

For  texture  generation  on  a  patch,  the  exact 
(u.v)  parametric  values  of  each  picture  element  on 
the  patch  are  used  to  look  up  a  texture  modulation 
table  as  in  Catmull's  algorithm. 

Whitted  Algorithm  (7).  Whitted's  algorithm 
is  similar  in  nature  to  Blinn's  but  avoids  the 
difficult  computation  involved  with  the  solution 
of  a  pair  of  bivariate  polynomial  functions  neces¬ 
sary  in  Blinn's  algorithm.  His  approach  is  to 
split  a  patch  into  smaller  subpatches.  If  the 
silhouette  curve  passes  through  a  subpatch  then 
the  segment  of  the  silhouette  curve  within  this 
subpatch  is  approximated  by  a  univariate  cubic 
curve  similar  in  form  to  a  boundary  curve. 
Therefore,  the  silhouette  curve  of  a  patch  is 
approximated  by  piecewise  cubic  curves.  Since 
the  boundary  curves  as  well  as  the  approximated 
silhouette  curves  are  now  univariate  cubic  curves, 
Whitted's  algorithm  only  has  to  deal  with  uni¬ 
variate  cubic  curves.  As  in  Blinn's  algorithm, 
Whitted  also  uses  the  Newton-Raphson  iteration 
technique  for  the  solution  of  univariate  cubic 
equations.  Both  algorithms  share  a  basic  problem: 
Although  their  numerical  iteration  technique  con¬ 
verges  quite  rapidly,  it  frequently  suffers  from 
stability  problems  associated  with  singularity 
points. 

Lane-Carpenter  Algorithm  (8).  The  Lane- 
Carpenter  algorithm  avoids  problems  associated 
with  numerical  iteration  techniques.  Their 
approach  is  to  employ  a  subdivision  algorithm 
similar  to  Catmull's.  However,  unlike  Catmull's 
algorithm,  subdivision  is  terminated  once  a  sub¬ 
patch  is  deemed  to  be  within  a  preset  tolerance  of 
being  planar,  at  which  time  the  four  corner  points 
of  the  subpatch  are  used  to  define  a  quadrilateral 
approximating  the  subpatch.  This  quadrilateral  is 
then  scan-converted  with  conventional  polygonal 
scan  conversion  algorithms 

Clark  Algorithm  (9) .  Clark's  algorithm  is 
similar  in  nature  to  the  Lane-Carpenter  algorithm. 
They  differ  mainly  in  the  mathematics  involved  in 
the  subdivision  and  the  test  for  planarity.  Clark 
uses  the  central  differencing  subdivision  tech¬ 
nique  which  is  faster  than  the  Bezier  (or  Bern¬ 
stein)  subdivision  technique  used  by  Lane  and 
Carpenter.  In  planarity  test,  Clark's  algorithm 
is  again  faster. 

The  advantage  of  the  Clark  and  Lane-Carpenter 
subdivision  algorithms  is  that  it  is  not  necessary 
to  consider  the  silhouette  curves  explicitly. 

In  addition,  the  planarity  test  is  based  on  the 
screen-space  curvature  of  the  subpatches;  conse¬ 
quently,  an  object  modeled  with  bicubic  patches  is 
essentially  being  approximated  dynamically  with  a 
number  of  quadrilaterals  depending  on  the  screen 
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sizes  of  the  patches.  The  number  of  quadrilaterals 
is  larger  when  the  object  is  close  than  when  it  is 
far  away.  The  minimum  number  of  quadri laterals  is 
equal  to  the  number  of  bicubic  patches  used  in 
defining  the  object.  This  has  the  effect  of  con¬ 
tinuously  changing  the  level-of-detai 1  polygonal 
representation  of  the  object. 

On  the  other  hand,  since  the  objects  eventu¬ 
ally  displayed  on  the  screen  are  approximated  by 
polygons,  the  usual  undesirable  effects  of  model¬ 
ing  objects  by  polygons  will  be  seen  on  images 
produced  by  the  Clark  and  Lane-Carpenter  algor¬ 
ithms.  Of  course,  if  the  tolerance  in  the  plan¬ 
arity  test  is  set  to  be  very  small,  then  a  smooth 
image  will  result.  However,  in  this  case  the 
number  of  polygons  to  be  scan-converted  will  be 
very  large. 

Inaba  Patches 

Inaba  patches  (3)  are  special  cases  of  bicubic 
patches.  Only  one  component  of  the  surface  defin¬ 
ition  of  an  Inaba  patch  is  bicubic;  the  other  two 
components  are  linear  in  one  of  the  two  independent 
variables  as  indicated  below: 

xe  =  u 

ye  =  v 

ze  =  UBZeBTVT 

Inaba  patches  are  appropriate  for  modeling 
terrain.  Their  major  advantage  over  the  more 
general  bicubic  patch  is  that,  while  the  surface 
normal  in  the  pilot's  eye  coordinate  system  of  a 
bicubic  patch  is  biquintic,  that  of  an  Inaba  patch 
is  bicubic.  Thus,  intensity  computation  using  the 
surface  normal  is  easier  in  Inaba  patches  than  in 
general  bicubic  patches. 

Biquadratic  Patches 

A  biquadratic  patch  is  a  parametric  patch 
whose  three  components  of  parametric  surface 
definition  are  biquadratic  in  two  independent 
variables  as  follows: 

xe  -  [u2  u  1  ]  BXgBT  U2  v  1  ] 

ye  =  [u2  u  1]  BYeBT  [v2  v  1] 

ze  =  [u2  u  1]  BZeBT  [v2  v  1] 

Xe,  Ye  and  Ze  are  3x3  matrices  defining  nine 
control  points.  B  is  a  constant  3x3  matrix. 

Because  of  the  lower  degree  of  the  polynomials 
involved,  biquadratic  patches  are  less  flexible 
for  modeling  free-form  curvilinear  objects,  and 
smoothness  across  patches  is  more  difficult  to 
maintain  than  the  bicubic  counterpart.  On  the 
other  hand,  the  computations  involved  for  their 
rendition  are  less  complicated. 

It  is  conceivable  that  one  could  apply 
Whitted's  approximation  technique  in  bicubic 
patches  to  biquadratic  patches  in  the  following 
manner.  A  biquadratic  patch  is  split  into  smaller 
subpatches.  If  the  silhouette  curve  passes 
through  the  subpatches,  it  is  approximated  by  a 
piecewise  univariate  quadratic  curve.  Since  now 
it  is  only  necessary  to  deal  with  univariate 


quadratic  curves,  the  task  of  finding  the  sil¬ 
houette  intersections  with  scan  lines  is  simpli¬ 
fied.  The  independent  variable  of  univariate 
quadratic  curve  intersections  with  scan  lines  can 
be  solved  either  iteratively  using  Newton-Raphson 
technique  or  analytically  by  the  solution  of  a 
quadratic  equation. 

QUADRIC  SURFACES 

Quadric  surfaces  are  surfaces  defined  by  a 
second  degree  equation  as  follows: 

alxe2  +  a2*e2  +  a3ze2  +  a4xeye  +  a5xeze  + 

a6>eze  +  a7xe  +  a8>e  +  a9ze  +  a10  *  0 

There  are  a  total  of  17  types  of  quadric  sur¬ 
faces  described  by  this  general  second  degree 
equation  (10).  Among  these  17  types  of  quadric 
surfaces,  five  are  imaginary  and  two  are  planar, 
and  so  only  10  of  them  are  real  curvilinear  sur¬ 
faces  and  are  useful  for  modeling  man-made 
curvilinear  objects  such  as  water  towers,  oil 
storage  tanks,  silos  and  other  aircrafts. 

Since  all  10  types  (except  real  ellipsoid)  of 
real  quadric  curvilinear  surfaces  have  infinite 
extent,  we  are  only  concerned  with  those  bounded 
by  planes  to  form  cylinders,  cones  and  frustums, 
etc.  Planar  ellipses  are  useful  for  modeling  bomb 
craters  and  other  man-made  features.  Moreover, 
the  bounding  of  real  quadric  surfaces  by  planes 
results  in  planar  ellipses.  Therefore,  we  con¬ 
sider  quadric  surface  generation  as  the  generation 
of  bounded  real  curvilinear  quadric  surfaces  and 
planar  ellipses.  These  quadric  surfaces  can  be 
combined  together  to  model  more  complicated  man¬ 
made  curvilinear  objects. 

It  can  be  shown  that  the  silhouettes  of  these 
quadric  surfaces  are  composed  of  either  straight 
lines  or  conic  sections.  There  exist  efficient 
algorithms  for  finding  silhouettes  of  these  quadric 
surfaces  as  well  as  their  intersections  with  scan 
lines  (11). 

An  efficient  scheme  for  intensity  computation 
for  quadric  surface  also  exists  (11).  This  inten¬ 
sity  computation  involves  the  incremental  genera¬ 
tion  of  two  quantities  and  their  subsequent 
division. 

PLANAR  SURFACES 

Because  of  the  ease  of  real-time  rendition  of 
planar  surfaces,  most  real-time  CGI  systems 
approximate  curvilinear  objects  by  planar  surfaces. 
The  si’houettes  of  planar  surfaces  are  easy  to 
generate  because  they  are  composed  of  straight 
lines. 

In  earlier  real-time  CGI  systems,  intensity 
for  each  planar  surface  was  constant  and  was  com¬ 
puted  by  calculating  the  dot  product  of  the  sun's 
illumination  direction  vector  with  the  surface 
normal  vector  of  the  planar  surface.  As  a  result, 
curvilinear  objects  looked  faceted.  To  eliminate 
the  faceted  appearance,  Gouraud  (12)  introduced 
an  algorithm  for  continuous  intensity  shading 
across  boundaries  of  planar  surfaces.  His  algo¬ 
rithm  involves  linear  interpolation  of  intensity 
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from  intensities  at  the  vertices  of  planar  poly¬ 
gons  and  is  quite  easy  to  implement  in  real  time. 
This  algorithm  improves  dramatically  the  appear¬ 
ance  of  curvilinear  objects  but  does  not  eliminate 
completely  the  faceted  appearance  because  even 
though  the  intensity  is  continuous,  the  intensity 
gradient  is  not.  Another  shortcoming  is  that  the 
silhouette  of  the  curvilinear  object  is  still 
composed  of  straight  edges. 

Phong  (13)  introduced  an  improvement  of 
intensity  computation  for  curvilinear  objects 
approximated  by  planar  surfaces  by  linearly  inter¬ 
polating  the  surface  normals  instead  of  intensi¬ 
ties  within  planar  polygons  from  those  at  the 
vertices  of  the  polygons.  The  linearly  interpo¬ 
lated  surface  normal  is  then  unitized  before  being 
used  for  intensity  computation.  Phong  also 
introduced  an  extra  term  in  the  intensity  illu¬ 
mination  model  to  include  specular  reflection 
which  adds  highlights  on  shiny  objects.  Because 
of  the  complex  computations  required,  no  real-time 
CGI  systems  have  incorporated  the  Phong  algorithm. 
Recently,  a  faster  implementation  of  the  Phong 
algorithm  has  been  independently  found  by  Yan  (11) 
and  Duff  (14). 

COMPARISON  OF  VARIOUS  CURVILINEAR 
OBJECT  GENERATION  TECHNIQUES 

Having  introduced  the  various  known  techniques 
for  computer  generation  of  curvilinear  objects,  we 
will  now  compare  them  in  terms  of  several  attri¬ 
butes  in  the  following  subsections. 

Ease  of  Generation  of  Silhouettes  of  Objects 

The  generation  of  silhouettes  of  curvilinear 
objects  Involves  first  the  findinq  of  the  defini¬ 
tions  of  the  silhouettes  and,  second,  the  finding 
of  their  intersections  on  scan  lines  given  their 
definitions.  The  following  lists  the  various 
curvilinear  object  generation  techniques  in  the 
order  of  decreasing  ease  of  generation  of  silhou¬ 
ettes. 

Planar  Surface  Technique.  The  silhouettes  of 
objects  approximated  by  planar  surfaces  are  com¬ 
posed  of  straight  lines  (first  degree  curves).  The 
definitions  of  these  straight  lines,  as  well  as 
their  intersections  on  scan  lines,  are  the  easiest 
to  obtain. 

Quadric  Surface  Technique.  The  silhouettes 
of  objects  modeled  with  this  technique  are  com¬ 
posed  of  conic  sections  (second  degree  curves). 
Efficient  algorithms  exist  to  find  the  definitions 
as  well  as  the  scan  line  intersections  of  these 
conic  sections. 

Biquadratic  Patch  Technique  with  Whitted's 
Approximation.  The  silhouettes  of  objects  pro¬ 
duced  with  this  technique  are  composed  of  uni¬ 
variate  quadratic  (second  degree)  curves. 

Algorithms  exist  to  find  their  intersection  on 
scan  lines.  However,  finding  the  definitions  of 
these  univariate  quadratic  curves  is  difficult 
and  Involves  the  splitting  of  patches  into  smaller 
subpatches  and  the  approximation  of  silhouette 
curves  by  piecewise  univariate  quadratic  curves. 

Lane-Carpenter  and  Clark  Bicubic  Patch  Tech¬ 
niques^  The  silhouettes  of  objects  eventually 
used  for  display  are  straight  lines  and,  therefore. 


their  intersections  on  scan  lines  are  easy  to  find. 
However,  the  finding  of  these  straight-line  sil¬ 
houettes  involves  expensive  subdivision  of  bicubic 
patches  and  tests  for  planarity. 

Whitted  Bicubic  Patch  Technique.  The  silhou¬ 
ette  s"^7_oEjicti-pro3uce3-wTtH_tffis  technique  are 
composed  of  univariate  cubic  (third  degree)  curves. 
Univariate  Newton-Raphson  iteration  technique  is 
necessary  to  solve  for  the  scan  line  Intersections 
of  these  silhouettes.  The  definitions  of  these 
univariate  cubic  curves  are  found  by  splitting 
bicubic  patches  into  smaller  subpatches  and  ap¬ 
proximation  of  silhouette  curves  by  piecewise 
univariate  cubic  curves. 

Blinn  Bicubic  Patch  and  Inaba  Patch  Tech¬ 
niques']  The  silhouettes  of  objects  produced  with 
these  techniques  are  composed  of  boundary  curves 
which  are  univariate  cubic  curves  and  silhouette 
curves  which  are  defined  by  a  pair  of  bivariate 
polynomial  functions.  Bivariate  (univariate) 
Newton-Raphson  iteration  technique  is  employed  to 
find  intersections  of  silhouette  (boundary)  curves 
with  scan  lines. 

Catmull  Bicubic  Patch  Technique.  Catmull's 
technique  differs  from  the  rest  of  the  techniques 
in  that  silhouettes  of  objects  are  implicitly 
obtained  when  a  patch  is  recursively  subdivided 
until  all  subpatches  cover  only  one  picture 
element.  This  process  is  expensive. 

Ease  of  Intensity  Computation  Within  Silhouettes 
of  Objects 

Intensity  computation  on  surfaces  of  objects 
within  the  silhouettes  involves  finding  the  uni¬ 
tized  surface  normals  of  the  surfaces  and  then 
performing  a  dot  product  of  the  sun's  direction 
vector  with  these  unitized  surface  normals. 
Curvilinear  object  generation  techniques  usually 
employ  approximations  to  the  above  computation  to 
speed  up  the  intensity  computation  process.  The 
following  lists  the  various  techniques  in  the 
order  of  decreasing  ease  of  intensity  computation. 

Planar  Surface  Technique.  Curvilinear  objects 
approximated  by  planar  polygons  have  a  unitized 
surface  normal  associated  with  each  vertex  of  the 
polygonal  approximations.  The  unitized  surface 
normal  at  each  vertex  approximates  closely  the 
surface  normal  of  the  real  curvilinear  objects. 

The  intensity  at  each  vertex  is  then  computed  and 
intensities  within  the  polygons  are  linearly 
interpolated  from  those  at  the  vertices.  Thus, 
intensity  computation  for  objects  modeled  with 
this  technique  is  quite  simple. 

Quadric  Surface  Technique.  For  quadric  sur¬ 
faces  tFii-surTacinormaTs_are_l  inear  in  the  screen 
xs,  ys  and  zs  coordinates.  This  allows  a  close 
approximation  of  surface  normal  within  the  sil¬ 
houette  of  a  quadric  surface  by  a  piecewise  linear 
approximation.  An  efficient  algorithm  exists  to 
perform  the  intensity  computation  using  the  piece- 
wise  linearly  interpolated  surface  normal.  This 
intensity  computation  involves  the  incremental 
generation  of  two  quantities  and  their  subsequent 
division. 

Parametric  Surface  Techniques ■  The  surface 
normal  as  well  as  the  surface  description  of  para¬ 
metric  surfaces  are  defined  on  the  same  two 
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independent  variables  (u,v).  To  compute  intensity 
exactly  at  each  picture  element  occupied  by  a 
visible  parametric  surface  patch,  it  is  necessary 
to  find  the  (u,v)  parametric  values  for  each  of 
these  picture  elements.  The  (u,v)  pair  is  then 
substituted  into  the  equations  of  the  surface 
normal  to  obtain  the  surface  normal  for  intensity 
computation.  This  is  quite  costly  in  processing 
time.  To  speed  up  intensity  computation  on  para¬ 
metric  surfaces,  the  surface  normals  and  intensi¬ 
ties  are  computed  exactly  only  at  certain  strategic 
points  within  parametric  surface  patches.  Inten¬ 
sities  at  points  other  than  these  strategic  points 
are  linearly  interpolated.  In  the  Clark  and  Lane- 
Carpenter  approach,  the  strategic  points  are  the 
corner  points  of  subpatches  which  are  deemed  close 
to  being  planar.  In  the  Blinn  approach,  there  are 
16  strategic  points  along  each  scan  line  covered 
by  a  patch.  In  the  Whitted  approach,  the  boundary 
curves  of  subpatches  and  the  approximated  silhou¬ 
ette  curves  are  strategic  points.  In  the  Catmull 
approach,  every  point  is  a  strategic  point. 

For  parametric  surface  techniques  such  as 
those  of  Catmull,  Clark,  and  Lane-Carpenter  which 
employ  subdivision  of  patches,  the  components  of 
surface  normal  are  subdivided  along  with  the  sur¬ 
face  patches.  Therefore,  no  substitution  of  (u,v) 
values  into  surface  normal  equations  is  necessary. 
However,  the  subdivision  of  surface  normal  is 
quite  expensive. 

Ease  of  Texture  Generation  Within  Silhouettes  of 
Objects. 

Texture  generation  on  an  object  is  the  pro¬ 
cess  of  mapping  a  two-dimensional  texture  modula¬ 
tion  pattern  onto  the  surface  of  the  object.  In 
order  to  have  the  texture  pattern  spatially  fixed 
to  the  surface,  each  point  on  the  surface  must  be 
associated  with  a  fixed  point  in  a  two-dimensional 
space  called  texture  mapping  space.  For  each 
visible  point  on  the  surface,  the  coordinates  of 
its  associated  point  in  the  texture  mapping  space 
are  used  to  look  up  a  two-dimensional  texture 
modulation  intensity  look-up  table.  The  texture 
modulation  intensity  looked  up  is  then  added  to 
the  intrinsic  intensity  of  the  surface. 

A  major  step  in  texture  generation  on 
a  surface  is  as  follows.  For  each  visible  picture 
element  of  the  surface,  find  the  coordinates  of 
its  associated  point  on  the  texture  mapping  space 
given  its  xs  and  ys  screen  coordinates  and  the 
geometry  of  the  surface.  We  will  refer  to  the 
above  step  as  inverse  transformation.  Texture 
generation  is  very  sensitive  to  error  in  the 
inverse  transformation  process;  a  slight  error  in 
performing  inverse  transformation  will  result  in 
texture  pattern  which  is  not  spatially  fixed  to 
the  surface  when  a  dynamic  sequence  of  texture 
generation  is  viewed.  Therefore,  inverse  trans¬ 
formation  has  to  be  accurately  performed  for  each 
visible  point  of  a  surface  which  is  to  receive 
texture;  no  coarse  approximation  to  the  inverse 
transformation  can  be  expected  to  work.  The  fol¬ 
lowing  lists  the  various  curvilinear  object  gener¬ 
ation  techniques  in  the  order  of  decreasing  ease 
of  performing  inverse  transformation  to  the  appro¬ 
priate  texture  mapping  space. 

Planar  Surface  Technique.  Since  each  point 
on  a  planar  surface  is  associated  with  a  fixed 
point  on  the  xe-ye  two-dimensional  space,  the 


xe-ye  space  is  an  appropriate  texture  mapping 
space  for  planar  surfaces.  One  major  computation 
involved  in  the  inverse  transformation  to  this 
texture  mapping  space  is  the  finding  of  zs  of  a 
point  on  the  planar  surface  given  its  xs  and  ys 
screen  coordinates.  It  can  be  shown  that  for 
planar  surfaces  zs  is  linear  with  xs  and  ys. 
Therefore,  z$  for  planar  surfaces  can  be  incre¬ 
mentally  generated. 

Quadric  Surface  Technique.  As  in  planar 
surfaces,  xe-ye  space  is  an  appropriate  texture 
mapping  space.  Again,  the  finding  of  z$  of  a 
point  on  the  quadric  surface,  given  its  xs  and  ys 
screen  coordinates  is  a  major  computation  of  the 
inverse  transformation.  However,  unlike  planar 
surfaces,  zs  for  quadric  surfaces  is  not  linearly 
related  to  xs  and  ys  and  its  evaluation  is  conse¬ 
quently  more  difficult. 

Parametric  Surface  Techniques.  Since  each 
point  on  a  parametric  surface  is  associated  with  a 
fixed  point  on  the  u-v  parametric  space,  the  u-v 
parametric  space  is  an  appropriate  texture  mapping 
space  for  parametric  surfaces.  To  perform  the 
inverse  transformation  for  each  and  every  visible 
picture  element  of  parametric  surfaces,  the  solu¬ 
tion  of  a  pair  of  bivariate  polynomial  functions 
is  required.  Thus,  texture  generation  on  para¬ 
metric  surfaces  is  quite  difficult. 

Image  Quality 

We  consider  two  aspects  of  image  quality  here 
The  first  is  the  smoothness  and  naturalness  of 
silhouettes  of  objects.  The  second  is  the  realism 
of  illumination  effect  rendition  within  the  sil¬ 
houettes.  The  following  lists  various  curvilinear 
object  generation  techniques  in  the  order  of 
decreasing  image  quality. 

All  Parametric  Surfaces  Techniques  Except 
Clark  and  Lane-Carpenter  Techniques.  Curvilinear 
objects  generated  with  these  techniques  have 
smooth  silhouettes.  Moreover,  since  information 
is  available  for  exact  calculation  of  surface 
normal  at  every  point  of  the  surfaces,  images  with 
extremely  realistic  illumination  effect  using 
sophisticated  illumination  models  can  be  generated 

Quadric  Surface  Technique.  Man-made  curvi- 
1 inear' objects  generated  with  this  technique  also 
have  smooth  silhouettes.  Moreover,  as  in  para¬ 
metric  surface  techniques,  information  is  avail¬ 
able  for  the  exact  calculation  of  surface  normal 
for  rendition  of  realistic  illumination  effect. 

Clark  and  Lane-Carpenter  Techniques.  Since 
curvilinear  objects  generated  with  these  tech¬ 
niques  are  eventually  approximated  by  planar 
polygons,  the  silhouettes  of  curvilinear  objects 
are  still  approximated  by  straight  lines.  However 
since  information  is  available  for  exact  calcula¬ 
tion  of  surface  normals  within  the  silhouettes, 
realistic  illumination  effect  can  be  generated 
within  the  silhouettes. 

Planar  Surface  Technique.  Tb«  silhouette  of 
curvilinear  objects  produced  with  this  technique 
are  composed  of  straight  lines.  Moreover,  sur¬ 
face  normals  are  available  only  at  the  vertices  of 
the  polygonal  approximation.  Thus,  a  surface 
normal  within  polygonal  surfaces  can  only  be 
approximated  by  Interpolation,  resulting  in  less 
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realistic  rendition  of  illumination  effect  than 
the  other  techniques. 

Data  Base  Compactness 

For  free-form  curvilinear  object  modeling, 
the  parametric  surface  approach  will  result  in  a 
more  compact  data  base  than  the  planar  surface 
approach.  The  quadric  surface  approach  is  not 
appropriate  for  modeling  free-form  curvilinear 
objects. 

For  man-made  curvilinear  objects  such  as  water 
towers,  silos,  oil  storage  tanks,  the  quadric  sur¬ 
face  approach  will  result  in  the  most  compact  data 
base.  The  parametric  surface  approach  will  result 
in  a  less  compact  data  base  and  the  planar  surface 
approach  will  result  in  the  least  compact  data 
base. 

Flexibility  of  Modeling  Free-Form  Curvilinear 
Objects 

The  parametric  surface  approach  is  by  far  the 
most  flexible  and  natural  for  modeling  free-form 
curvilinear  objects.  Images  generated  with  this 
approach  have  proven  to  be  smooth  and  natural . 

Among  the  different  techniques  within  the  para¬ 
metric  approach,  those  using  higher  order  poly¬ 
nomials  will  be  more  flexible  in  modeling  free¬ 
form  curvilinear  objects  than  lower  order  ones. 
Therefore,  bicubic  patches  are  the  most  flexible, 
Inaba  patches  are  less  flexible,  and  biquadratic 
patches  are  the  least  flexible  for  modeling  free¬ 
form  curvilinear  objects. 

The  planar  surface  approach  is  quite  flexible 
for  modeling  free-form  curvilinear  objects. 
Unfortunately,  images  generated  with  this  approach 
have  undesirable  unnatural  artifacts  discussed 
above. 

The  quadric  surface  approach  is  the  least 
flexible  for  modeling  free-form  curvilinear 
objects. 

CONCLUSION 

Three  approaches  to  computer  generation  of 
curvilinear  objects  have  been  surveyed.  These 
three  approaches  are  compared  in  terms  of  six 
attributes.  The  planar  surface  approach  is  first 
in  three  out  of  the  six  attributes,  namely,  ease 
of  generation  of  silhouettes,  ease  of  intensity 
computation  and  ease  of  texture  generation.  Tex¬ 
ture  generation  provides  effective  motion  and 
attitude  cues  for  pilot  training.  We  feel  that 
planar  surface  approach  will  continue  to  prevail 
in  real-time  CGI  systems  for  flight  training. 

The  quadric  surface  approach  is  second  in 
ease  of  silhouette  generation,  ease  of  intensity 
computation,  ease  of  texture  generation  and  image 
quality.  It  is  first  in  data  base  compactness 
for  modeling  man-made  curvilinear  objects.  We 
feel  that  quadric  surface  generation  with  smooth 
shading  (intensity  computation)  but  without  tex¬ 
ture  generation  is  a  viable  addition  to  real-time 
CGI  systems  to  augment  planar  surface  generation 
with  smooth  shading  and  texture  generation. 

For  curvilinear  objects,  the  parametric  sur¬ 
face  approach  is  first  in  image  quality,  data  base 
compactness,  and  flexibility  of  modeling  free-form 


curvilinear  objects.  Unfortunately,  it  is  sub¬ 
stantially  more  difficult  than  the  other  two 
approaches  in  silhouette  generation,  intensity 
computation,  and  texture  generation.  Parametric 
surface  approach  will  prevail  in  applications 
where  image  quality  is  important  and  real-time 
generation  is  not  important.  Such  an  application 
is  animation  for  television  commercials. 
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JOINING  TECHNIQUES  FOR  OPTICALLY  COMBINED  VISUAL  DISPLAY  SYSTEMS 
Jerry  L.  Bentz 

McDonnell  Douglas  Electronics  Company 
St.  Charles,  MO 


Since  the  introduction  and  FAA  certification 
of  VITAL  in  March,  1972,  computer  generated  image 
display  modules  containing  a  spherical  mirror, 
beamsplitter,  and  cathode  ray  tube  (CRT)  have 
become  very  familiar  to  flight  simulator  visual 
system  users.  Increased  utilization  of  simulators 
has  created  the  need  for  expanded  visual  system 
fields  of  view.  CGI  techniques  and  creative  varia¬ 
tions  to  these  basic  display  modules  are  satisfying 
this  need. 

In  less  than  one  decade,  visual  display  sys¬ 
tems  have  progressed  from  single  module,  single 
channel  to  multiple  channels  using  optically  com¬ 
bined  modules.  To  provide  for  these  broader  re¬ 
quirements,  McDonnell  Oouglas  Electronics  Company 
(MDEC)  has  developed  three  separate  approaches 
employing  optical  combining  techniques:  Modular, 
Wide  Field  of  View  and  Zero  Gap.  All  are  capable  of 
being  optically  joined  into  multiple  module  config¬ 
urations  with  overlapping  imagery,  and  each  ex¬ 
hibits  a  different  set  of  tradeoff  considerations. 
This  paper  discusses  field  of  view  (FOV),  and  de¬ 
scribes  the  joining  characteristics  for  each  tech¬ 
nique.  It  concludes  by  presenting  photographs  of 
imagery  from  our  latest  configurations,  the  Wide 
Field  of  View  and  Zero  Gap  overlapped  displays. 

FIELD  OF  VIEW 

The  multichannel  systems  which  began  to  emerge 
in  1973  necessitated  the  coordination  of  the  sev¬ 
eral  modules  around  the  cockpit.  Determination  of 
module  location  requires  a  careful  analysis  of 
training  tasks  and  almost  always  involves  judge¬ 
mental  compromises.  Considerations  must  include 
modular  structural  constraints,  cockpit  geometry 
and  head  motion.  Suppliers  of  visual  systems  must 
also  recognize  there  are  significant  mission  and 
head  motion  differences  between  military  fighters 
and  commercial  aircraft.  Regardless  of  aircraft 
type,  however,  an  understanding  of  FOV  characteris¬ 
tics  is  necessary  to  determine  which  of  the  three 
optical  module  joining  techniques  is  best  suited 
for  a  particular  visual  system. 

Total  FOV  and  instantaneous  FOV  are  two  terms 
used  in  discussing  visual  display  systems.  Al¬ 
though  widely  used,  there  are  no  generally  accepted 
definitions  for  these  terms.  At  MDEC,  we  use  the 
following  first  order  definitions  for  horizontal  or 
vertical  maxima: 

Total  Field  of  View,  (TFOV)  = 

-1 

2(Sin. )  [  maximum  CRT  dimension 

I  2(0ptical  system  focal  length)  J 

Instantaneous  Field  of  View,  (IFOV)  * 

2(Sin.)  r  maximum  mirror  dimension 
I  2 ( Eye  to  mirror  distance) 

Equating  these  terms  to  the  real  world  situa¬ 
tion,  TFOV  is  comparable  to  the  outside  world  scene 


and  IFOV  is  comparable  to  a  window  frame  through 
which  the  outside  world  is  viewed.  TFOV  and  IFOV 
may  be  obstructed  by  module  side  panels  or  beam¬ 
splitter  supports,  by  intrusion  of  the  CRT,  or  by 
aircraft  structure. 


The  location  of  the  IFOV  changes  in  the  azi¬ 
muth-elevation  coordinate  system  with  head  motion. 
Depending  upon  the  exact  hardware  configuration  and 
amount  of  observer  head  movement,  the  IFOV  may  not 
be  large  enough  to  continue  the  visibility  of  some 
portion  of  the  TFOV.  To  help  visualize  these  com¬ 
plex  FOV  relationships,  MDEC  developed  a  computer 
program,  with  various  plotting  options,  to  cor¬ 
rectly  illustrate  the  relationship  of  mirror,  CRT, 
beamsplitter  and  TFOV  in  an  azimuth-elevation  co¬ 
ordinate  system.  The  two  plots  shown  in  Figure  1 
demonstrate  the  effects  of  observer  head  motions, 
in  this  example  7.5  inches  left  and  3  inches  down 
from  nominal,  on  a  single  module  located  “straight 
ahead"  of  the  pilot.  The  plotting  option  used  in 
the  lower  half  of  the  Figure  clearly  shows  that 
only  a  portion  of  the  CRT  is  visible  from  the  ex¬ 
treme  viewing  position. 


Each  of  the  three  joining  techniques  for  com¬ 
bining  optical  modules  establishes  specific  rela¬ 
tionships  between  T  and  IFOV's. 

JOINING  TECHNIQUES 

The  new  systems  now  being  used  demanded  new 
solutions  to  old  problems.  Several  forms  of  elec¬ 
trical  distortion,  for  example,  become  critical  and 
can  limit  system  performance.  However,  these  pro¬ 
blems  have  been  solved  separately  and  are  readily 
separable  from  the  purely  optical  considerations 
involved  in  expanding  system  FOV's.  In  this  paper 
we  will  concentrate  on  the  optical  characteristics 
at  the  module  edges  of  these  three  distinct  methods 
of  joining  displays  with  overlapping  imagery. 

THE  MODULAR  OPTICS  TECHNIQUE 

The  term  “Modular  Optics"  refers  to  a  family 
of  optical  assemblies  each  containing  a  spherical 
mirror  and  beamsplitter.  Modules  have  a  36°  x  48° 
TFOV.  The  difference  between  the  modules  is  mainly 
in  the  size  of  horizontal  IFOV  as  shown  in  Figure 
2.  All  modules  have  side  panels  (opaque  edges). 
Where  modules  provide  overlapped  imagery,  adjacent 
sides  represent  a  common  cut  through  the  center  of 
a  sphere.  TFOV  horizontal  half-angles  exceed  cor¬ 
responding  IFOV  half-angles  by  1°,  resulting  in  a 
TFOV  overlap  of  2°.  These  characteristics  allow 
modules  to  be  positioned  side-by-side  with  freedom 
to  have  relative  vertical  offsets,  a  major  distinc¬ 
tion  and  a  frequently  significant  operational  ad¬ 
vantage  over  the  Zero  Gap  technique  to  be  described 
later.  Monocularily,  beamsplitter  joints  between 
modules  subtend  about  1/5°.  With  binocular  view¬ 
ing,  no  full  length  beamsplitter  joining  gaps  are 
seen  from  the  reference  position. 
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FIGURE  1.  SAMPLE  PLOTS  SHOWING  EFFECTS  OF  HEAD  MOVEMENT 
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Note  that,  with  one  exception,  FOV  sketches, 
as  well  as  the  photographs  of  the  displays  which 
follow  later,  generally  represent  monocular  view¬ 
ing:  There  are  just  too  many  lines  on  plots  repre¬ 
senting  binocular  viewing  for  easy  comprehension. 
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FIGURE  2.  MODULAR  OPTICS  PACKAGE  VARIATIONS 

The  effects  of  head  movement  on  joined  Modular 
Optics  are  shown  in  Figure  3,  which  illustrates  two 
viewing  positions  of  a  generalized  three  module 
display  system.  Binocular  viewing  from  the  nominal 
head  position  is  represented  at  the  junction 
between  right-hand  and  center  modules.  Viewing  is 
essentially  gap-free.  Binocular  viewing  from  left 
of  the  nominal  head  position  is  represented  at  the 
junction  between  left-hand  and  center  modules. 
Oblique  viewing  of  the  side  panels  is  occluding  a 
portion  of  the  center  window  imagery.  All  of  the 
left  module's  imagery  near  the  overlap  region  is 
seen.  With  Modular  Optics,  head  movement  from  the 
nominal  head  position  is  evidenced  by  the  black 
band  developing  between  display  modules.  Black 
band  growth  occurs  at  a  rate  of  about  1  3/4°  per 
inch  of  head  movement. 

An  interesting  additional  variation  to  this 
display  package  concept  is  illustrated  in  Figure  4. 
This  Vertical  Package  module  is  now  being  applied 
to  several  situations  in  which  the  quarter  display 
has  a  vertical,  rather  than  horizontal,  orientation 
to  permit  increased  up-viewing  (as  in  banked  turns) 
and  down-viewing  (runway  edges  during  landing 
phase)  with  one  display  package.  This  optical 
package  was  developed  to  eliminate  severe  beam¬ 
splitter  reflection  problems  from  cockpit  lighting 
which  may  occur  with  a  simple  90°  rotation  of  a 
conventional  optical  module.  One  tradeoff  in  use 
of  this  optical  package  is  a  slightly  larger,  3° 
gap  between  adjacent  displays. 

Modular  Optics  have  proven  quite  useful  in  meeting 
a  wide  range  of  training  requirements.  This  is 
particularly  true  in  situations  for  which  relative 
vertical  offset  between  adjacent  modules  is  impor¬ 
tant.  These  packages  have  been  implemented  with  a 
variety  of  geometric  parameters  to  satisfy  differ¬ 
ent  operational  requirements:  For  example,  we  have 
incorporated  spherical  mirrors  with  radii  of  49 
1/2,  58  and  80  inches  in  various  systems. 


THE  WIDE  FIELD  OF  VIEW  TECHNIQUE 

The  Wide  Field  of  view  (WFQV)  optical  system 
represents  a  significant  departure  from  the  classic 
36°  x  48°  module  because  it  provides  a  TFOV  of  60°  x 
45°  in  a  single  module.  Similar  electronic  assem¬ 
blies  including  the  same  CRT  are  used  for  WFOV  and 
Modular  Optics.  The  larger  TFOV  is  obtained  by 
using  a  different  mirror  radius  and  a  multielement 
relay  lens  to  shorten  the  optical  system  focal 
length. 

This  is  a  pupil-forming  system  with  a  9  inch 
diameter  pupil.  As  shown  schematically  in  Figure 
5,  the  system  can  be  packaged  with  the  60°  axis 
vertical,  and  this  large  vertical  subtension  makes 
relative  vertical  offsets  between  adjacent  modules 
less  important.  With  zero  relative  elevation  dif¬ 
ference  between  modules,  the  adjacent  module  beam¬ 
splitters  intersect  in  a  line  subtending  the  same 
azimuth  angle  as  the  edges  of  the  module  mirrors, 
and  side  panels  can  be  removed.  An  8°  TFOV  overlap 
between  modules  provides  full-pupil  viewing.  Vig¬ 
netting  within  the  relay  lens  occurs  only  at  the 
extreme  corners  of  Lhe  display  field.  Beamsplit¬ 
ters  for  the  60°  vertical  orientation  are  long  (5 
feet)  and  require  substantial  (1  inch  wide)  sup¬ 
ports.  Except  for  a  small  diamond  shaped  area,  the 
beamsplitter  supports  do  not  block  the  binocular 
image. 

The  primary  advantages  of  the  WFOV  are  its  60° 
x  45°  TFOV  and  its  ability  to  provide  essentially 
gap-free  overlapping  TFOV's  for  binocular  viewing 
with  head  motion.  One  anticipated  disadvantage  of 
the  WFOV  optical  system  was  its  9  inch  diameter 
pupil.  However,  experience  has  now  shown  that  in  a 
single  seat  fighter  cockpit  viewing  application 
this  pupil  size  is  completely  acceptable.  The 
first  three  module  installations  using  WFOV  are  in¬ 
stalled  and  operational  .  The  principal  application 
for  these  systems  is  for  air-to-air  intercept  and 
weapons  delivery  training.  A  three-module  FOV  dia¬ 
gram  and  the  visual  system  are  shown  in  Figure  6. 

THE  ZERO  GAP  TECHNIQUE 

The  advantage  of  relative  vertical  offset  cap¬ 
ability  offered  by  Modular  Optics  is  sometimes 
overshowed  by  the  desire  to  remove  any  gap  at  all 
between  adjacent  displays.  For  this  purpose  a  Zero 
Gap  optical  joining  technique  was  developed. 

Zero  Gap  optical  modules  differ  from  Modular 
Optics  primarily  in  the  amount  of  TFOV  overlap  and 
in  removal  of  the  side  panels  between  adjacent  dis¬ 
plays.  CRT  shape  and  practical  choices  of  mirror 
radius  limit  the  overlap  to  a  maximum  of  about  8°. 
There  can  be  no  relative  vertical  offset  between 
adjacent  displays  or  the  beamsplitter  planes  would 
not  have  the  line  intersection  necessary  to  allow 
elimination  of  the  side  panels.  Beamsplitter  di¬ 
mensions  are  small  enough  that  gap-producing  beam¬ 
splitter  supports  are  not  required.  A  large  single 
mirror  is  used  for  two-module  Zero  Gap  displays  to 
preclude  mirror  alignment  errors  and  mirror  radius 
mismatch,  both  possible  with  use  of  two  mirrors. 

The  principal  Zero  Gap  viewing  benefit  is  gap¬ 
less  viewing  with  head  motion,  up  to  the  limits  of 
overlap.  Implemented  with  the  full  8°  overlap, 
allowable  head  movement  before  a  gap  develops  is  5 
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FIGURE  3.  TWO  BINOCULAR  VIEWS  OF  JOINED  MODULAR  OPTICS 
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14-1557  6  (1215-1M9) 

FIGURE  6.  THE  JA37  VISUAL  SYSTEM 
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inches  for  binocular  viewing  and  3  3/4  inches  for 
monocular  viewing.  Figure  7  illustrates  the  condi¬ 
tion  which  exists  for  head  movement  of  5  1/4 
inches,  the  position  where  a  gap  begins.  The  tri¬ 
angular  gap  will  enlarge  with  additional  head  mo¬ 
tion  to  the  left,  at  a  rate  of  about  1  3/4°  per  inch 
of  movement.  The  primary  limitation  of  the  Zero 
Gap  joining  technique  is  inability  to  provide  rela¬ 
tive  elevation  offsets  across  the  joined  displays. 

SYSTEM  PERFORMANCE 

Many  combinations  of  display  modules  are  pos¬ 
sible.  Figure  8  summarizes  some  recent  VITAL  IV 
installations  using  up  to  seven  display  modules  per 
cockpit.  Multiwindow  display  systems  with  narrow 
or  zero  gaps  between  adjacent  modules  must  have 
good  alignment  error  correction  for  acceptable  sys¬ 
tem  performance.  Display  CRT  and  related  electron¬ 
ic  deflection  errors  which  must  be  reduced  are 
shown  in  Figure  9. 

Alignment  procedures  have  been  specifically 
developed  for  these  new  overlapped  display  systems. 


These  procedures,  conducted  by  the  simulator  tech¬ 
nician  using  only  electrical  adjustments,  have 
proven  effective  in  achieving  the  required  reduc¬ 
tion  of  electrical/optical  errors. 

Development  of  The  Zero  Gap  optical  joining 
technique  was  begun  at  MD EC  in  1975.  Only  re¬ 
cently,  however,  have  we  obtained  the  improved 
electrical  components  and  mirrors  of  adequate  qual¬ 
ity  required  for  this  type  display.  Consistent, 
high  quality  mirror  production  is  now  being 
achieved  in  our  new  mirror  fabrication  facility. 

CONCLUSION 

As  a  result  of  some  innovative  but  straight¬ 
forward  optical  design  work,  impressively  large 
field  of  view  coverage  is  now  appearing  in  visual 
systems.  Large  field  of  view,  alone,  no  longer 
dictates  high-cost  visual  equipment.  With  these 
display  solutions  now  available,  military  and  com¬ 
mercial  training  Program  Managers  have  more  flex¬ 
ibility  to  satisfy  broader  demands  in  training 
through  simulation. 


FIGURE  7.  EFFECTS  OF  EXTREME  HEAD  MOTION  ON  ZERO  GAP  MODULES 
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FIGURE  9.  POTENTIAL  CRT  DEFLECTION  ERRORS 

The  following  photographs  illustrate  how  well  system  distortion  errors  are  controlled  with  today's  tech¬ 
nology. 
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ABSTRACT 


The  main  specifications,  for  the  visual  displays  associated  with  military  air¬ 
craft  simulators  are  first  recalled  ;  they  take  Into  account  the  eye  proper¬ 
ties  for  spatial  resolution,  colour  and  speed  of  response.  These  specifica¬ 
tions  are  examined  from  the  point  of  view  of  simulator  design  and  video-to- 
visual  image  converters.  The  features  of  a  new  projector  developed  by  SODERN 
are  then  briefly  described,  and  their  adaptation  to  simulator  needs  are  dis¬ 
cussed. 


THE  SIMULATION  NEEDS 

During  the  first  In  ter serv ice/ 1 ndustry 
Training  Equipment  Conference,  the  lec¬ 
tures  and  discussions  (1)  have  proven  the 
Importance  of  the  simulation  of  visual 
stimuli  on  the  training  equipment  effi¬ 
ciency  ;  the  relative  Importance  of  the 
visual  parameters  of  the  display  :  reso¬ 
lution,  colour...  has  been  discussed. 

For  military  aircraft  simulators,  the 
most  demanding  requirement  is  to  cover 
more  than  three  quarters  of  the  full 
space  (i.e.  10  steradians  of  solid 

angle)  around  the  pilot  with  a  high  reso¬ 
lution  image.  The  eye  resolution  capabil¬ 
ity  decreases  very  fast  with  the  angular 
distance  to  the  foveal  direction.  At 
field  center,  in  daylight  level  or 
photopic  condition,  the  limit  of  visibi¬ 
lity  of  a  black  disc  on  a  clear  uniform 
background  corresponds  to  a  25  arc  sec¬ 
onds  diameter,  i.e.  12  nsr  (nanostera- 
dians),  but  in  the  6  sr  eye  field,  the 
observer  may  see  1  million  picture  ele¬ 
ments  (pixels)  only,  which  correspond  to 
an  average  solid  angle  at  6  usr  (micro- 
steradians)  each.  The  eyeball  motion  in 
the  orbit,  the  head  rotation  capability, 
and  the  use  of  rear-view  mirrors  extend 
the  use  of  the  high  resolution  capability 
at  eye  field  centre  everywhere  in  the 
space  around  the  observer.  Therefore,  the 
simulator  demand  for  resolution  is  of  the 
order  of  10  sr/12  nsr  =  830  millions  of 
picture  elements. 

The  image  rate  also  is  an  Important  para¬ 
meter  :  military  scenes  are  usually  fast 
moving  ones,  due  either  to  the  targets 
speed  (ground  vehicles,  boats,  aircrafts, 
rockets)  or  still  more  the  angular  motion 
of  the  trainee  vehicle  :  for  the  display, 
the  image  angular  motion  to  be  reproduced 
may  be  of  radians  per  second  order  of 
magnitude.  From  this  point  of  view,  the 


most  critical  difference  between  the  nat¬ 
ural  scenes  and  their  reproduction  by 
cinematographic  (movie  or  television) 
means  is  that  the  reproduced  ones  are 
time  sampled  :  objects  moving  with  re¬ 
spect  to  the  reproduction  frame  have  a 
jerky  motion  the  eyes  do  not  follow.  :Je 
shall  come  back  on  this  point.  Far  i o  !•»>-- 
ate  angular  speed,  the  demand  is  to  avoid 
the  flicker  effect  which  appears  at  image 
rates  lower  than  15  to  30  per  second  de¬ 
pending  on  duty  cycle,  light  level  and 
location  in  the  field.  Higher  rates  are 
recommended  for  long  duration  observa¬ 
tions  . 

For  the  naked-eye  observer,  the  image 
luminosity  is  characteri zed  by  the  lumi¬ 
nance  of  the  light  flow  which  reaches  his 
eyes  :  colour  vision,  spatial  resolution 
and  in  a  lesser  extent  reaction  speed  to 
visual  stimuli  depend  on  this  parameter  : 

-  the  effect  on  colour  vision  is  well 
known  :  the  change  from  the  photopic 
colour  vision  to  the  scotopic  colour¬ 
less  one  occurs  in  the  10  to  0,01 
lm.m-2.sr-l  (or  3  to  0,003  foot- 
-Lambert)  luminance  range, 

-  the  effect  on  spatial  resolution  is 
Illustrated  by  figure  1  which  gives  the 
number  of  just  resolved  line  pairs  per 
radian  in  the  eye  foveal  region  versus 
the  clear  lines  luminance  (dark  lines 
luminance  =  0). 

The  visual  characteri zation  of  the  image 
Is  not  limited  to  the  spatial  and  time 
resolutions  ;  at  a  given  time  and  for 
each  pixel,  the  luminance  and  colour  dif¬ 
ferential  steps  levels  are  also  Important 
parameters  : 

-  the  luminance  dynamics  should  reach  a 
range  from  e.g.  100  to  1,  with  a  mini¬ 
mum  of  64  steps  (6  bits  or  a  maximum  of 
7%  averaqe  difference  from  step  to 
step) , 

-  the  reproduced  colours  range  should 
cover  most  of  the  natural  colours.  The 
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THE  SIMULATORS  CAPABILITY 


~1  jure  l 

Reciprocal  R  of  the  line  pairs  period  of 
a  black  and  white  periodical  pattern  just 
resolved  by  human  eyes,  as  a  function  of 
white  lines  luminance  L. 

****** 

National  Television  Standards  Committee 
(NTSC)  conventional  range  selected  for 
the  broadcast  TV  (fig.  2)  covers  about 
45  %  of  the  discernable  colours  at  pho- 
topic  luminance  levels  (and  a  much 
higher  percentage  of  natural  ones).  For 
a  given  luminance  level,  about  2500 
different  colours  may  be  recognized, 
thus  about  1000  in  the  NTSC  range,  this 
corresponds  to  a  10  bits  chrominance 
range  ;  a  3  hits  chrominance  discrimi¬ 
nation  could  probably  be  sufficent. 

Assuming  830  million  pixels  per  field, 
30  fields  per  second,  6  bits  luminance 
range  and  8  bits  chrominance,  the  visual 
display  input  rate  should  be  3.5.10  to 
the  11th  bits  per  second. 

V 


Figure  2 

Uniform  chromaticity  plane,  NTSC  conven¬ 
tional  triangle  (RGB)  and  colour  range  as 
reproduced  by  the  three  light  valves 
SODERN  projector  (R 'G'B  1  ) . 

****** 


The  capabilities  of  real  simulators  are 
lii'ii  t.ed  by  technology  and  cost  :  trade- 
-offs  have  to  be  found  to  produce  the 
most  acceptable  Images  talcing  into  ac¬ 
count  the  characteristics  of  the  avail¬ 
able  technologies  and  the  relative  im¬ 
portance  of  the  image  qualities  which 
have  been  described  in  the  previous  chap¬ 
ters  . 

Trainee  head  motion  allowance  and  particularly 
the  displays  for  two  or  three  people  team  require 
large  optical  extent  (product  of  the  front  area 
of  visibility  by  the  field  angle).  For  wide 
field  angles,  two  families  of  displays  are  used: 

-  real  image  displays  where  the  trainee 
heads  are  close  from  the  center  of  a 
large  hollow  sphere  or  done  on  the  in¬ 
ner  wall  of  which  a  mosaic  of  images  is 
projected , 

-  virtual  image  displays  where  the  train¬ 
ee  head  is  close  of  the  center  of  a 
dodecahedron  of  joining  wide  field  op¬ 
tics  (pancake  windows*)  at  the  front 
focal  surfm.e  of  which  are  produced  the 
real  image  of  the  local  field. 

For  the  same  requirement  at  the  trainee 
level,  both  designs  do  not  demand  the 
same  performances  from  the  optical  image 
generator  :  luminous  output,  field  angle, 

subraster  capability .  The  aim  of  this 

paper  is  not  to  compare  these  designs, 
but  to  analyze  the  adaptability  of  the 
video-to-optical  image  converters  to 
their  needs. 

The  wide  field  dome  displays  require  high 
output  image  generators  ;  due  to  the  mul¬ 
tiple  reflection  inside  the  dome,  a 
trade-off  has  to  be  made  between  sphere 
diameter,  scene  luminance  and  sphere  con¬ 
trast  losses  when  the  projector  output  is 
limited.  Virtual  image  displays  introduce 
geometrical  implantation  constraints 
which  make  of  great  Interest  the  intro¬ 
duction  of  subraster  capabilities,... 
We  shall  not  go  in  futher  details,  the 
above  ones  were  only  to  show  the  inter¬ 
action  between  the  display  design  and  the 
v 1 deo- to-v i sual  image  converter  capabili¬ 
ties. 

These  last  capabilities  are  limited  by 
thp  physical  phenomena  used  for  the  image 
conversion.  Competing  solutions  use  : 

-  cathode  ray  tubes  in  which  the  spatial¬ 
ly  modulated  light  is  produced  by  a 
single  component  :  this  solution  is 
limited  in  luminous  output,  colour 
range  (for  a  single  tube)  and  screen 
size.  Spatial  resolution  may  be  high. 


(*  a  trade-mark  of  Farrand  Optical 
Company) . 
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-  equipment  In  which  the  light  is  pro¬ 
duced  and  modulated  by  separate  com¬ 
ponents  ;  three  types  of  light  modula¬ 
tors  (or  light  valves)  are  now  develop¬ 
ed  : 

-  using  refraction  or  diffraction  by 
oil  films, 

-  using  polarization  in  solid  crystals, 

-  using  polarization  In  liquid  crys¬ 
tal  s  , 

all  able  of  much  higher  output,  with 
limited  resolution  capability  and  dif¬ 
ferent  time  dependent  characteristics. 

Presently,  the  spatial  resolution  of  the 
light  valves  is  limited  to  about  1  mil¬ 
lion  pixels  per  frame,  thus  the  use  of  a 
mosaic  of  e.g.  10  images  each  covering  1 
steradian  field  is  usually  accepted  :  the 
corresponding  1  usr  pixel  size  (4,5  arc 
minute  disc  diameter)  is  to  be  compared 
to  the  12  nsr  wish  :  artifices  are  to  be 
found  to  enhance  the  spatial  resolution 
in  the  field  where  it  is  really  needed. 

It  is  known  that  for  a  given  spatial  res¬ 
olution  in  complex  scenes,  the  detection 
and  identification  capabilities  are  in¬ 
creased  when  luminance  levels  (grey 
shades)  and  colour  data  are  present  : 
this  increases  the  favour  of  colour  dis¬ 
plays  when  spatial  resolution  needs  can 
not  be  fully  satisfied. 

The  video-to-visual  image  converters  time 
behaviour  is  also  an  important  feature  : 
the  time  distribution  of  the  light  in  a 
frame  period.  Too  small  of  a  duty  cycle 
increases  the  flicker  effect.  If  the 
image  erasing  time  is  too  long,  the  image 
becomes  laggy,  smearing  details  in  moving 
scenes,  a  very  important  fact  in  simul¬ 
ation. 


THE  SODERN  VIDEO  IMAGE  PROJECTOR 

The  SOOERN  video-to-visual  image  convert¬ 
er  is  of  the  light  valve  type  (2),  it 
uses  a  white  light  xenon  arc  as  light 
source,  and  electron  beam  addressed  light 
valves  using  polarization  in  a  solid 
crystal  (more  details  about  light  valve 
and  projector  operation  in  annex).  The 
images  produced  by  this  type  of  modulator 
are  characterized  b: 

-  large  luminous  flux  control  capability 
(more  than  2500  lumens), 

-  large  pixels  rate  capability  (more  than 
3,5.10  the  7th  per  second), 

-  neither  flicker  nor  lag  (simultaneous 
erasing  and  writing), 

-  image  storage  time  capability  up  to 
minutes , 

-  subraster  capability, 

-  flat  image  (no  apparent  line  struc¬ 
ture)  , 

-  large  distortion  correction  capability 
(scanning  rates), 

-  random  access  capability. 


Light  valves  and  projectors  using  this 
principle  have  been  manufactured  at  labo¬ 
ratory  scale  during  the  years  1976-77  for 
interactive  cable  television  experiments 
in  large  movie  theaters.  The  main  charac¬ 
teristics  of  this  equipment  are  discussed 
in  annex  1.  This  equipment  has  been  put 
in  operation  many  times  in  various  places 
since  their  manufacture,  and  are  still  in 
operation  with  their  original  light  val¬ 
ves:  they  have  proven  to  be  reliable, 
with  low  maintenance  and  adjustment. 

Most  of  the  above  mentioned  image  features 
have  been  demonstrated*. 

The  development  of  a  version  adapted  to 
the  simulation  and  data  display  is  now  in 
progress,  partly  supported  by  US  and 
French  Government  agencies.  This  develop¬ 
ment  takes  into  account  of  the  most  com¬ 
mon  requirements ,  for  functional,  inter¬ 
face  and  environmental  specifications, 
but  also  for  availability,  ma 1 n tenab il 1 ty 
and  life  cycle  cost  points  of  view.  In 
this  development  work,  the  emphasis  is 
put  on  the  improvement  of  the  resolution 
since  it  is  Vi*  most  critical  performance 
for  the  contemplated  applications.  From 
our  point  of  view,  the  unconventional 
features  of  the  images  produced  by  this 
type  of  light  valve  give  new  possibili¬ 
ties  for  the  optimization  of  the  image 
generation  system,  which  are  not  fully 
expl ored . 


*  a  movie  illustrating  the  main  image 
features  (absence  of  flicker,  absence  of 
lag,  image  flatness,  resolution,  storage 
capability)  is  presented  at  the  Confer¬ 
ence. 
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Annex 

THE  SODERN  VIDEO  IMAGE  PROJECTOR 
PRINCIPLE  AND  CHARACTERISTICS 

The  schematic  diagram  of  the  equipment  Is 
represented  by  figure  3. 


SODERN  light  valve  projector  structure 
( schemati c ) . 


AS  =  aperture  stop,  BLV  =  blue  channel 
light  valve,  BRM  =  blue  reflecting  mir¬ 
ror,  CFL  =  coll Imating-focusing  lens,  EM= 
ellipsoidal  mirror,  FC  =  focus  converter, 
FL  =  field  lens,  F$  =  field  stop,  GLV  = 
green  channel  light  valve,  PA  =  polar¬ 
izer-analyzer  ,  RLV  »  red  channel  light 
valve,  RRM  =  red  reflecting  mirror,  XA  * 
xenon  arc. 

****** 


The  light  produced  by  a  xenon  arc  is  col¬ 
lected  and  concentrated  by  an  elliptical 
mirror  on  a  field  stop.  An  image  of  this 
field  stop  is  focused  by  a  set  of  lenses 
and  through  a  polarizer  on  the  electro- 
-optical  target  of  three  light  valves. 
An  aperture  stop  is  located  at  the 
polarizer  exit  and  at  the  front  focus  of 
col  1 imatl ng-foc usi ng  lenses.  The  white 
beam  is  split  in  three  spectral  bands, 
the  blue,  green  and  red  light  beams  re¬ 
flected  back  by  the  three  light  valves 
are  recombined  and  then  cross-back  the 
polarizer  which  is  now  used  as  an  analyz¬ 
er.  The  target  of  the  light  valve  behaves 
as  a  birefringent  plane  mirror,  the  local 
birefringence  of  which  is  electrically 
modulated  according  to  the  video  signal. 
The  coll  imati ng-focusing  lenses  are  used 


to  focus  the  targets  image  onto  the 
screen.  This  combination  of  locally  mod¬ 
ulated  birefringence,  polarizers  and  Fo¬ 
cusing  optics  provides  an  Intensity  mod¬ 
ulated  image  onto  the  screen  for  each 
channel.  Due  to  use  of  a  single  exit  pu¬ 
pil,  the  registration  of  the  three  super¬ 
imposed  colour  image  components  is  not 
affected  by  the  shape  and  position  of  the 
screen . 

In  order  to  understand  the  features  of 
the  Images  produced  by  the  light  valve  it 
is  necessary  to  say  few  words  about  the 
light  valve  target  operation  (figure  4). 


Figure  4 

Titus  light  valve  electro-optical  target 
structure  and  mode  of  operation.  DM  * 
dielectric  mirror,  E  *  electrons,  EOH  = 
electro-optical  wafer,  G  »  grid,  L  * 
light,  Te  =  transparent  electrode,  VIE  * 
video  input  electrode. 

****** 


The  electro-optical  component  of  this 
target  is  a  thin  (0,2  mm)  wafer  of  a 
transparent  dielectric  crystal  the  bire¬ 
fringence  of  which,  in  the  direction  per¬ 
pendicular  to  the  plane  faces,  depends  on 
the  electrical  field  applied  in  this  di¬ 
rection  (longitudinal  Pockels  effect). 
The  Illuminated  face  (right  side  on  the 
figure)  is  coated  with  a  transparent  and 
conductive  layer  used  as  input  electrode 
for  the  video  signal.  The  opposite  face 
(left  side)  has  received  a  set  of  alter¬ 
nate  low-high  refractive  index  dielectric 
layers  which  behaves  as  a  mirror.  The 
outer  surface  of  this  mirror  is  scanned 
by  a  constant  voltage  electron  beam  ;  the 
design  of  the  electron  tube  part  of  the 
light  valve  is  such  as  this  electron  beam 
may  remove  as  well  as  to  supply  electrons 
up  to  give  the  surface  the  same  voltage 
as  the  constant  voltage  of  a  grid  placed 
nearby . 
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If  the  electron  beam  current  is  high  e- 
nough  to  supply  the  target  with  all  the 
electrical  charge  per  unit  area  corre¬ 
sponding  to  the  product  of  the  target 
capacity  by  the  peak  video  voltage,  when 
scanning  the  target  surface  the  electron 
beam  replaces  the  previous  charge  status 
by  a  new  one  corresponding  to  the  instan¬ 
taneous  video  voltage.  As  the  target  and 
the  mirror  are  made  of  good  dielectric 
materials,  the  charge  remains  unchanged 
up  to  the  new  scanning  :  this  explains  : 

-  the  target  linage  storage  capability  and 

-  the  absence  of  flicker  and  of  lag. 

The  local  target  polarization  property  is 
voltage  controlled  :  as  far  as  the  elec¬ 
tron  beam  current  density  is  high  enough, 
when  the  video  voltage  is  transferred 
through  the  target  the  supplied  extra 
charges  are  rejected  :  this  explains  : 

-  the  image  flatness  without  resolution 
loss  and 

-  the  absence  of  influence  of  the  elec¬ 
tron  beam  scanning  speed  on  the  lumi¬ 
nous  output. 

The  combination  of  the  two  above  de¬ 
scribed  facts  explains  also  why  parts 
only  of  the  image  can  be  refreshed  (sub¬ 
raster  and  calligraphic  modes). 

The  equipment  manufactured  in  1976-77  has 
been  designed  to  fit  the  European  broad¬ 
cast  standards  : 

-  25  images  per  second  (50  frames,  double 
interl ace) , 

-  625  lines  per  frame, 

650  pixels  per  line  at  modulation 
transfer  factor  (MTF)  =  5  i. 

The  colour  reproduction  range  is  deter¬ 
mined  by  the  dichroTc  beam  splitter  cut 
off  wave  lengths  : 

-  red  channel  :  595  to  660  nm, 

-  green  :  515  to  575  nm, 

-blue  :  440  to  490  nm, 

these  primaries  are  represented  in  the 
constant  chromaticity  plane,  figure  2. 

The  luminous  output  for  a  all  white  image 
reaches  2700  lumens,  the  dynamic  range 
(all  clear/all  dark  fields  output)  reach¬ 
es  60/1. 

Local  deviation  of  illuminance  and  colour 
in  the  field  are  well  inside  the  profes¬ 
sional  movie  specification. 

The  new  light  valves  ( manuf ac tured  in  the 
frame  of  a  contract  directed  by  USAF 
through  General  Electric)  give  now  20  % 
FTM  at  650  pixels  per  line. 


(1  list  Interservice/Industry  Training 
Equipment  Conference,  Orlando  Fla., 
27-29  Nov.  1979 

-  Panel  discussion  :  Simulator  techn¬ 
ology,  trends  and  forecasts. 

-  Workshop  :  On  the  training  effec¬ 
tiveness  of  colour  for  military 
flight  training  simulation. 


(2)g.  Marie,  "Light  valves  using  DKDP 
operated  near  its  Curie  point":  Titus 
and  Phototitus  -  Ferroelectri cs  1976, 
Vol.  10,  pp  9-14.  Gordon  and  Breach 
Science  Publishers  Ltd.,  London  1976. 


64 


BEHAVIORAL  BASES  FOR  DETERMINING  VEHICLE  DETAILING  IN  SIMULATION  DISPLAYS 


Dr.  Brian  L.  Kottas,  Research  Psychologist 
US  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
Fort  Knox,  Kentucky 
and 

Dr.  David  W.  Bessemer,  Simulation  Systems  Team  Chief 
US  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
Fort  Knox,  Kentucky 

ABSTRACT 


Effective  representation  of  armored  vehicles  in  simulation  displays  demands  a  careful 
evaluation  of  human  perceptual  capabilities.  This  holds  especially  true  for  computer  gen¬ 
erated  target  displays,  which  must  provide  sufficient  detail  to  allow  vehicle  identification 
within  limitations  of  computer  processing  time  and  display  resolution.  Even  in  image  gen¬ 
eration  and  display  systems  not  incurring  such  limitations,  the  image  detail  need  not  exceed 
human  perceptual  and  cognitive  information  processing  capabilities.  Providing  excessive 
detailing  of  targets  may,  in  fact,  produce  negative  training  by  allowing  those  being  trained 
to  depend  on  information  unavilable  in  combat  for  target  identification.  Results  of  re¬ 
search  manipulating  visible  target  detail  in  target  identification  training  and  its  implica¬ 
tions  for  target  displays  are  discussed.  Estimates  are  presented  for  the  visibility  of 
features  of  threat  and  friendly  main  battle  tanks,  based  on  analysis  of  past  empirical  re¬ 
search  done  under  ideal  visibility  conditions  and  visibility  data  from  past  research. 


INTRODUCTION 

Picture  yourself  as  a  Tank  Commander,  moving 
in  column  along  a  road  toward  your  forward  battle 
position.  Suddenly,  you  detect  two  tracked  vehi¬ 
cles  moving  along  a  treeline  about  2000  meters 
away.  Do  you  engage  them,  or  don't  you?  You  have 
only  split  seconds  to  decide.  If  they  are  friend¬ 
ly  and  either  simply  lost  or  moving  to  a  position 
near  yours,  engaging  them  would  result  in  the 
death  of  US  soldiers  or  your  allies.  If  they  are 
enemy,  failing  to  engage  them  endangers  your  life, 
tne  lives  of  your  men,  and  your  tank.  Are  they 
friend  or  foe?  This  question  will  recur  frequent¬ 
ly  given  the  fluid  and  "dirty"  battlefield  ex¬ 
pected  in  most  future  European  scenarios. 

The  situation  described  above  clearly  points 
out  the  need  for  effective  target  identification 
training.  Simulation  offers  exciting  opportuni¬ 
ties  for  training  many  aspects  of  combat  perfor¬ 
mance,  including  target  identification,  since  it 
affords  the  capability  to  present  a  wide  variety 
of  realistic  targets  in  highly  realistic  tactical 
settings.  Troops  can  be  trained  to  identify  tar¬ 
gets  within  the  normal  context  of  maneuver  and 
combat  engagement.  However,  the  capability  to 
train  target  identification  as  a  matter  of  course 
raises  the  question  of  how  much  detail  computer¬ 
generated  vehicle  representations  or  other  target 
displays  require  to  allow  realistic  target  identi¬ 
fication.  The  detail  must  be  sufficient  to  repre¬ 
sent  a  vehicle's  critical  features  (those  that 
differentiate  it  from  other,  similar  vehicles), 
but  the  display  should  not  represent  features  that 
would  be  unavailable  in  a  combat  setting  and  at 
the  ranges  a  target  would  normally  be  identified, 
The  danger  with  highly  detailed  representations  Is 
that  troops  might  learn  to  depend  on  target  de¬ 
tails  that  are  simply  unavailable  for  target  iden¬ 
tification  in  combat. 

METHOD 

The  US  Army  Research  Institute  (ARI)  field 
unit  at  Fort  Knox  approached  the  target  represen¬ 
tation  problem  for  tank  gunnery  by  examining  ob¬ 
servers'  performance  in  learning  to  Identify 
slides  of  tank  targets  at  different  ranges,  both 


before  and  after  an  experimental  ARI  target  identi¬ 
fication  training  program  (see  reference  2). 
Simulated  training  and  test  ranges  of  4000  and  2000 
meters  through  the  8x  sight  of  the  M60A1  were  se¬ 
lected,  because  these  two  ranges  encompass  the 
ranges  in  which  tank  target  identification  would 
normally  take  place,  assuming  good  visibility  con¬ 
ditions  and  making  two  other  reasonable  assumptions. 
The  first  assumption  is  that  few  targets  in  a  tac¬ 
tical  setting  will  be  detected  beyond  4000m  (with 
the  exception  of  large  armor  formations,  in  which 
case  target  identification  is  a  different  task  than 
for  individual  vehicles),  and  the  second  is  that  an 
enemy  should  be  identified  at  a  range  of  2000m  or 
more,  given  the  engagement  hit  probabilities  given 
in  FM  71-1  (1).  Examining  observers'  ability  to 
learn  target  identification  at  these  two  ranges 
allows  one  to  determine  how  much  of  a  difference, 
if  any,  in  target  identification  training  occurs 
due  to  reduced  visibility  of  target  features  at  the 
farther  compared  to  the  nearer  range.  From  these 
data  one  can  infer  the  detail  needed  in  displays  to 
train  target  identification  at  ranges  demanded  in 
combat. 


Subjects 


Twenty  0SUT  (One  Station  Unit  Training)  armor 
crewmen  (10  gunner/loaders  and  10  drivers)  served 
as  observers.  All  had  completed  the  standard  block 
of  instruction  on  Soviet  Soldiers  and  Equipment, 
which  includes  basic  target  identification  training, 
prior  to  the  experimental  target  identification 
training  program  received  as  a  part  of  this  re¬ 
search.  In  this  paper,  however,  references  to 
training  refer  to  training  received  through  the  ex¬ 
perimental  training  program  administered  to  sub¬ 
jects  in  this  research. 


Stimuli  and  Apparatus 


Targets  for  training  and  tests  of  observers' 
learning  were  presented  via  slides.  Slides  of  ve¬ 
hicles  were  taken  from  the  ARI  Combat  Vehicle  Iden¬ 
tification  (CVI)  Program  (2),  as  were  the  verbal 
descriptions  of  the  vehicles  that  were  presented 
during  the  training  program.  The  training  program 
Itself  was  a  modified  version  of  one  module  of  the 
ARI  Combat  Vehicle  Identification  Program,  and 
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lasted  approximately  two  to  two  and  one-half  hours. 
Seven  tanks  were  selected  for  training  and  testing, 
with  the  rationale  that,  first,  tanks  are  the  pri¬ 
mary  targets  for  tanks  on  the  battlefield,  and 
second,  that  they  are  highly  confusable  combat  ve¬ 
hicles  and  therefore  present  some  of  the  most  dif¬ 
ficult  discriminations  to  be  made  in  target  iden¬ 
tification  in  combat.  The  seven  tanks  used  were 
the  Soviet  T-55,  T-62,  and  T-72,  the  French  AMX30, 
the  US  M60A1 ,  the  West  German  Leopard  I,  and  the 
British  Chieftain.  The  slides  showed  model  vehi¬ 
cles  on  a  terrain  board.  All  vehicles  were  photo¬ 
graphed  on  the  same  spot  on  the  terrain  board,  and 
all  vehicles  were  camouflage  painted.  These  meas¬ 
ures  prevented  observers  from  learning  to  identify 
vehicles  based  on  extraneous  cues  from  the  terrain 
or  vehicle  markings.  Frontal,  flank,  and  oblique 
views  of  all  vehicles  were  included.  Slides  of 
vehicles  were  rear-projected  by  a  Kodak  Ektagraphic 
slide  projector.  Model  AF-1.  Exposures  of  slides 
were  timed  by  a  hand-held  stopwatch. 

Procedure 

Observers  were  tested  and  trained  in  groups 
of  ten.  Within  each  group  of  ten,  five  observers 
were  selected  to  receive  the  experimental  target 
identification  training  program  at  a  simulated 
range  of  2000m;  the  other  five  observers  received 
the  training  program  at  a  simulated  range  of  4000m. 
Ranges  were  simulated  by  seating  the  observers  at 
different  distances  from  the  screen  upon  which 
slides  were  projected  at  a  constant  scale  (for 
more  detail  see  the  instructions  and  discussion 
accompanying  the  CVI  package).  Five  gunner/ loaders 
received  training  at  2000m,  and  five  at  4000m;  the 
same  held  true  for  the  drivers. 

Before  undergoing  the  experimental  training 
program,  each  observer  was  given  two  tests  (without 
feedback)  on  all  views  of  all  vehicles,  with  one 
pretest  at  each  of  the  two  ranges.  Presentation  of 
the  slides  of  all  seven  vehicles  in  each  of  three 
views  was  randomized  for  each  pretest.  Each  slide 
in  each  of  the  21-item  pretests  was  presented  for 
ten  seconds  and  followed  by  a  ten  second  period 
with  a  blank  screen  to  allow  the  observers  more 
time  to  write  down  their  answers.  After  the  pre¬ 
tests,  the  vehicle  identification  program  was  pre¬ 
sented.  This  program  included  verbal  descriptions 
of  vehicles  accompanying  the  slides  of  the  models, 
as  well  as  slides  of  vehicles  that  observers  were 
asked  to  name,  and  for  which  feedback  was  given 
following  their  responses.  Specific  details  of 
the  training  program  are  presented  in  an  ARI  Report 
currently  in  preparation.  After  the  instruction, 
all  observers  were  given  two  final  tests,  with  one 
test  at  2000m  and  one  at  4000m.  As  with  the  pre¬ 
tests,  order  of  presentation  was  randomized  over 
all  vehicles  and  all  views  on  both  tests. 

RESULTS  AN0  DISCUSSION 

Analysis  of  variance  was  applied  to  the  num¬ 
ber  of  correct  vehicle  Identifications  (corrected 
for  guessing).  The  analysis  revealed  that  perfor¬ 
mance  improved  significantly  after  the  experimental 
training  program  (p<.01).  Figure  1  shows  this  ef¬ 
fect  graphically  In  terms  of  percent  correct,  cor¬ 
rected  for  guessing.  The  difference  between  per¬ 
formance  at  simulated  ranges  of  2000m  and  4000m 
was  not  statistically  reliable  either  before  or 
after  subjects  received  the  experimental  training 
program.  Similarly,  the  difference  between 
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Figure  1.  PERFORMANCE  IMPROVEMENT  WITH  ARI  COMBAT 
VEHICLE  IDENTIFICATION  PROGRAM 

observers  trained  at  2000m  and  those  trained  at 
4000m  was  insignificant,  as  was  the  interaction  be¬ 
tween  training  range  and  test  range.  The  above  re¬ 
sults  show  that  first,  the  ARI  Combat  Vehicle  Iden¬ 
tification  Program  raises  target  identification 
performance  significantly  over  that  provided  by 
Armor  0SUT  training,  regardless  of  some  variation 
in  training  and  testing  range,  and  second,  that 
there  is  still  more  room  for  improvement. 

The  analysis  also  showed  that  performance  dif¬ 
fered  for  different  vehicle  orientations  (p< . 01 ) . 
Figure  2  shows  this  effect  graphically,  with  results 
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Figure  2.  PERFORMANCE  DIFFERENCES  FOR  DIFFERENT 
VEHICLE  ORIENTATIONS 

before  and  after  the  experimental  training  program 
combined.  One  must  qualify  statements  about  this 
effect,  however,  due  to  a  significant  Interaction 
of  vehicle  orientation  with  training.  Figure  3 
shows  this  interaction.  A  test  of  simple  main  ef¬ 
fects  revealed  first,  that  performance  after  train¬ 
ing  Improved  significantly  over  that  before  training 
for  all  three  vehicle  orientations  (p< .01  in  all 
three  cases),  and  second,  that  performance  differ¬ 
ences  among  different  vehicle  views  were  significant 
only  after  training  (p< . 01 ) .  Pairwise  comparisons 
using  Tukey's  HSD  revealed  that  after  training, 
observers  identified  flank  views  of  vehicles  sig¬ 
nificantly  better  than  both  frontal  views  (p<.01) 


Figure  3.  PERFORMANCE  DIFFERENCES  FOR  DIFFERENT 
VEHICLE  ORIENTATIONS  BEFORE  AND  AFTER 
TRAINING 


and  oblique  views  (p< . 01 ) .  These  comparisons  sim¬ 
ply  show  that  subjects  in  the  present  research 
learned  the  most  about  identifying  flank  views  of 
vehicles,  and  the  least  about  identifying  frontal 
views  of  vehicles.  The  performance  differences  be¬ 
tween  different  vehicle  orientations  simply  re¬ 
flects  the  differences  in  the  number  of  critical 
features  available  in  the  three  views.  Training 
observers  to  identify  frontal  and  oblique  vehicles 
present  a  particular  challenge  for  future  target 
identification  training  research,  since  the  ten¬ 
dency  to  fire  at  another  vehicle  without  making  a 
positive  identification  first  will  be  greatest  for 
vehicles  in  those  orientations. 

The  striking  aspect  of  these  results  is  that, 
within  practical  limits,  the  effects  of  the  range 
at  which  observers  were  trained  and  tested,  as  well 
as  the  interaction  of  these  two  factors,  were  ex¬ 
tremely  small  relative  to  the  large  improvement  of 
all  observers  over  training,  and  were  not  even  re¬ 
liable  enough  to  approach  statistical  significance. 
This  implies  that,  for  all  practical  purposes,  ob¬ 
servers  learned  to  identify  targets  just  as  well 
with  the  details  that  could  be  seen  at  4000m 
through  an  8x  sight  as  they  could  with  the  details 
that  could  be  seen  at  2000m  through  an  8x  sight. 
Therefore,  when  considering  training  observers  to 
identify  tank  targets  through  the  8x  M60A1  sight 
under  ideal  visibility  conditions,  one  need  not 
have  as  much  detail  as  can  be  seen  normally  at 
2000m,  although  including  that  much  detail  will  not 
harm  training. 

One  might  wonder  why  insignificant  effects  of 
training  and  test  range  were  found  in  the  above  re¬ 
search.  This  question  can  be  answered  by  first 
considering  data  collected  by  Foskett,  et  al.  (4) 
on  the  detectability  of  features  of  armored  vehi¬ 
cles.  Figure  4,  taken  from  their  report,  shows  the 
detectabilities  of  the  features  they  investigated. 
Consider  that  ranges  of  2000  and  4000  meters 
through  an  8x  sight  correspond  approximately  to  250 
and  500m  with  unaided  vision  if  the  atmosphere  Is 
perfectly  clear.  One  can  see  by  Figure  4  that 
there  Is  only  a  small  difference  between  the  de¬ 
tectability  of  features  at  250  and  500  meters  with 
unaided  vision.  Small  features,  such  as  the  number 
of  roadwheels  (If  the  number  Is  large)  become  dra¬ 
matically  less  detectable  over  this  Increase  in 
range,  but  the  probability  of  detecting  the  largest 
features  drops  very  little.  Coupling  this  rela¬ 


tively  small  difference  in  detectability  with  the 
poor  performance  before  training  and  the  fact  that 
the  ARI  program  emphasizes  Identification  using 
large,  highly  noticeable  features  (while  de-empha¬ 
sizing  the  importance  of  small  features)  it  is 
probably  not  surprising  that  the  effects  of  train¬ 
ing  or  test  range  were  insignificant. 

In  considering  the  results  of  the  above  re¬ 
search,  however,  there  is  a  very  definite  caveat. 
Both  our  research  and  the  research  of  Foskett  et  al. 
were  conducted  under  scaled  conditions.  Hence, 
they  apply  specifically  to  an  atmosphere  of  almost 
infinite  vis4  iity  and  do  not  consider  effects  of 
atmospheric  attenuation.  The  results  of  the  re¬ 
search  reported  here  are  an  overestimate  of  the 
actual  detectability  of  vehicle  features  in  a  nor¬ 
mal  outdoor  environment.  The  impact  of  various 
levels  of  atmospheric  attenuation  on  actual  detec¬ 
tion  of  armored  vehicle  features  is  yet  to  be  thor¬ 
oughly  investigated.  However,  one  can  develop  a 
crude  estimate  of  the  effects  of  atmospheric  atten¬ 
uation  through  data  such  as  those  presented  by 
Middleton  (5).  Middleton  presented  a  series  of 
nomograms  allowing  one  to  predict  the  95  percent 
detection  level  of  objects  of  various  sizes,  given 
‘certain  levels  of  background  luminance,  the  target 
to  background  contrast  prior  to  contrast  extinction 
by  the  atmosphere,  and  meteorological  range  (that 
range  at  which  atmospheric  attenuation  reduces  tar¬ 
get  to  background  contrast  to  threshold).  The  data 
Middleton  presented  allows  one  to  generate  curves, 
such  as  those  in  Figure  5,  from  which  one  can  esti¬ 
mate  the  amount  of  drop  in  visibility  of  a  feature 
with  reduced  lighting  and  reduced  meteorological 
range.  For  more  detail  on  the  derivation  of  Figure 

5,  the  Interested  reader  is  referred  to  a  forthcom¬ 
ing  ARI  Working  Paper  dealing  with  vehicle  feature 
detectability  (6).  If  one  assumes  that  these 
curves  can  be  considered  a  "pure"  reduction  in 
visibility,  disregarding  any  interactions  of  visi¬ 
bility  reduction  with  target  background,  one  can 
rescale  the  abscissa  of  Figure  4  to  reflect  a  cer¬ 
tain  proportion  of  drop  in  detection  range.  An 
example  of  this  rescaling,  showing  the  impact  of 
reduced  visibility  and  a  reduced  light  level  on  the 
detection  data  of  Foskett  et  al.  is  shown  in  Figure 

6.  It  is,  of  course,  only  a  very  rough  estimate, 
and  actual  likelihoods  of  detection  in  a  tactical 
setting  depend  on  a  number  of  factors,  including 
such  things  as  the  target  background  and  observers' 
criteria. 

One  can  see  from  Figure  6  that  under  very  hazy 
and  overcast  conditions,  features  become  much  more 
difficult  to  see.  Only  gross  features  such  as 
whether  a  vehicle  is  tracked  or  wheeled,  whether  or 
not  it  has  fender  skirts,  its  main  gun,  its  turret 
position,  and  turret  shape  are  readily  available. 

Any  difference  in  target  identification  at  4000m 
and  2000m  under  these  conditions  is  yet  to  be  de¬ 
termined.  For  a  further  discussion  of  the  impact 
of  atmospheric  conditions  on  feature  visibility, 
the  Interested  reader  is  again  referred  to  the 
forthcoming  ARI  paper  on  feature  visibility  (6). 

What  conclusions  can  be  drawn  from  the  research 
reported  here?  Since  the  lethality  of  modern  weap¬ 
ons  may  easily  make  2000m  the  minimum  range  by 
which  an  enemy  tank  must  be  identified  under  good 
viewing  conditions,  it  seems  impractical  to  conduct 
target  identification  training  with  a  final  goal  of 
training  people  to  identify  tanks  only  within  2000 
meters.  Therefore,  for  practical  purposes  it  seems 
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Figure  4.  DETECTION  RANGES  FOR  SELECTED  VEHICLE  FEATURES  (FIGURE  FROM  FOSKETT,  BALDWIN,  AND  KUBALA,  1978) 


Figure  5.  DETECTION  RANGE  AS  A  FUNCTION  OF  METE¬ 
OROLOGICAL  RANGE  FOR  CIRCULAR  TARGET  (AREA  =  I  SQ. 
FT.)  AGAINST  SKY  BACKGROUND.  CURVES  FIT  BY  HAND. 


unnecessary  to  train  target  identification  with 
more  details  than  would  be  visible  under  ideal 
viewing  conditions  through  an  8x  sight  at  2000m. 
Additionally,  since  any  difference  between  target 
identification  training  at  2000  and  4000m  proved 
to  be  small  relative  to  other  influences  in  the 
present  research,  only  those  that  can  be  seen  at 
4000  meters  with  an  8x  sight  are  necessary  to  in¬ 
clude  for  practical  training  purposes.  Since  this 
research  did  not  address  training  with  even  fewer 
features  than  those  available  at  4000  meters  are 
necessary  for  reliable  identification,  until  fur¬ 
ther  research  is  conducted. 

Several  questions  yet  remain  to  be  answered 
that  may  have  an  Impact  on  future  target  identifi¬ 
cation  training  programs.  What  is  the  impact  of 
including  more  features  than  are  available  at 
2000m?  Is  it  feasible  to  consider  beginning  target 
identification  training  with  highly  detailed  vehi¬ 
cle  representations,  and  systematically  reduce  the 
detailing  until  vehicles  can  be  identified  based  on 
minimal  information?  Finally,  a  better  understand¬ 
ing  of  the  Impact  of  time  constraints  during  target 
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Range  (In  Meter*  X  lOo) 

Figure  6.  ESTIMATED  DETECTION  RANGES  FOR  VEHICLE  FEATURES  SELECTED  BY  FOSKETT  ET  AL.  UNDER  OVERCAST 
CONDITIONS,  WITH  5  KM  VISIBILITY 


identification  and  the  impact  of  observers'  cri¬ 
teria  may  yield  information  about  practical  aspects 
of  target  identification  that  have  yet  to  be  con¬ 
sidered. 
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ABSTRACT 


Aircraft  simulators  have  in  the  past  been  designed  and  used  primarily  as  substitutes  for 
aircraft.  CGI  provides  the  flexibility  to  enhance  training  in  ways  not  normally  possible  in  an 
aircraft.  This  research  conceived  and  demonstrated  training  approaches  to  take  advantage  of 
this  flexibility.  Rather  than  duplicate  the  real  world  cockpit  environment,  this  study  con¬ 
siders  other  objectives  attainable.  This  requires  a  change  from  thinking  of  a  simulator  as  an 
airplane  substitute  to  thinking  of  it  as  a  training  device  that  can  complement  'real  world' 
training  from  text  through  actual  aircraft.  Another  change  requiring  consideration  is  the 
expected  change  in  military  use  of  simulators  from  teaching  initial,  simple  flight  skills  and 
procedures,  to  teaching  and  maintaining  complex  combat  skills  involving  interactions  among 
several  aircraft  and  ground  systems. 

Scene  elements  were  incorporated  into  a  computer  generated  visual  presentation  which  did 
not  represent  "real  world"  objects,  but  which  were  there  solely  for  instructional  purposes. 

Hopefully  these  techniques  will  be  used  to  improve  the  effectiveness  of  actual  aircraft 
hours  spent  in  training  rather  than  replace  them. 


INTRODUCTION 


In  the  past,  aircraft  simulators  have  been  de¬ 
signed  and  used  primarily  as  substitutes  for  actual 
aircraft.  Computer  generated  imagery  provides  the 
flexibility  to  enhance  training  in  ways  that  cannot 
be  done  in  an  aircraft,  at  least  under  peacetime 
regulations.  The  thrust  of  this  research  is  to 
conceive  and  demonstrate  new  training  approaches  to 
take  advantage  of  this  flexibility  as  a  step  to¬ 
wards  reducing  pilot  combat  attrition  and  in¬ 
creasing  readiness.  Two  broad  categories  of  tech¬ 
niques  are  available  to  us:  (1)  simulation  of 
tasks  untrainable  in  aircraft  during  peacetime  but 
required  during  combat,  and  (2)  application  of 


teaching/learning  methods  unavailable  in  aircraft. 
Examples  of  the  first  category  are  surface-to-air 
missile  (SAM)  avoidance  and  air-to-air  missile 
avoidance.  This  category  was  not  emphasized,  since 
many  of  these  techniques  are  incorporated  in  pre¬ 
sent  state-of-the-art  visual  systems  or  will  be  in 
the  near  future.  The  second  category  is  exempli¬ 
fied  by  techniques  such  as  making  visible  something 
that  the  pilot  must  visualize  but  cannot  see  in  the 
real  world  (e.g.,  a  radar  antenna  pattern  of  an 
opposing  aircraft  during  air-to-air  combat  or  a 
diagram  of  the  relative  energy  state  of  the  air¬ 
craft  and  their  flight  envelopes,  or  allowing  the 
student  to  view  an  engagement  from  various  view¬ 
points  (his  own,  ground  threats',  air  threats', 
overview,  etc.  (See  Figures  1-3). 


FIGURE  1.  VISIBLE  THREAT  CONE 

*  Research  sponsored  by  the  Air  Force  Office  of  Scientific  Research  (AFSC).  United  States  Air  Force,  under 
Contract  F49620-79-C-0067.  The  United  States  Government  is  authorized  to  reproduce  and  distribute 
reprints  for  governmental  purposes  notwithstanding  any  copyright  notation  hereon. 
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FIGURE  2.  RELATIVE  FLIGHT  ENVELOPE  PERFORMANCE  DIAGRAM 
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OBJECTIVE 

The  objective  was  to  conceive  and  demonstrate 
examples,  such  as  described  above,  of  concepts  in 
aircrew  training  that  would  take  advantage  of  the 
flexibility  of  computer  generated  imagery  to  pro¬ 
vide  enabling  and  instructional  features  unavail¬ 
able  in  the  “real  world".  The  emphasis  was  in 
complex  combat  skill  training  as  opposed  to  simple 
tasks  such  as  takeoff  and  landing. 


APPARATUS 

Some  of  the  concepts  resulting  from  this  ef¬ 
fort  have  been  demonstrated  on  a  VITAL  IV  computer 
generated  image  (CGI)  system,  located  at  McDonnell 
Douglas  Electronics  Company.  This  system  consists 
of  a  general  purpose  minicomputer,  special  purpose 
high  speed  computational  hardware,  and  a  calli¬ 
graphic  color  display  with  collimating  optics  as 
shown  in  Figure  4.  The  display  is  normally  mounted 
outside  the  window  of  an  aircraft  simulator  cockpit 


to  display  a  representation  of  the  "real  world"  to 
the  pilot.  The  simulator  position  and  attitude 
information  in  the  response  to  the  pilot's  controls 
is  fed  into  the  visual  system  which  updates  the 
scene  correspondingly  thirty  times  per  second. 
Typical  scenes  are  shown  in  Figure  5.  The  scenes 
are  made  according  to  customer  desires  and  have  in 
the  past  been  made  specifically  to  simulate  the 
real  world  flight  environment  including  special  ef¬ 
fects  such  as  variable  weather  conditions,  surface- 
to-air  missiles,  air-to-air  missiles,  anti-air¬ 
craft  artillery,  tracer  bullets,  and  so  forth. 

All  of  these  scenes  are  made  up  using  two  ba¬ 
sic  elements,  flat  convex  polygonal  shapes  and 
strings  of  light  points.  There  are  no  restrictions 
however  on  how  one  can  use  these  basic  scene  ele¬ 
ments.  In  particular,  they  need  not  be  used  to 
make  scenes  to  look  like  the  "real  world."  It  is 
the  objective  of  this  project  to  explore  these 
other  uses  to  which  the  basic  scene  elements  (com¬ 
mon  to  many  computer  generated  image  systems)  may 
be  put. 


FIGURE  4.  VITAL  III  SYSTEM  COMPONENTS 


IDEA  GENERATION  PROCESS  AND  RESULTS 

To  provide  a  framework  for  the  idea  generation 
process  a  list  of  key  issues  to  be  trained  was 
drawn  up  (Table  1). 

Simultaneous  with  the  generation  of  specific 
training  techniques  a  list  of  generic  techniques 
was  created  to  avoid  getting  into  a  rut  of  essen¬ 


tially  similar  approaches.  In  the  next  year's  work 
the  idea  generation  process  will  continue  but  will 
have  the  added  benefit  of  feedback  from  preliminary 
trials. 

GENERIC  TECHNIQUES 

A  list  of  generic  training  techniques  is  given 
in  Table  2.  Each  will  be  discussed  in  the  para¬ 
graphs  that  follow.  These  are  not  all  claimed  to 
be  new,  nor  is  the  list  comprehensive. 


73 


AIRCRAFT  CARRIER  WITH 
SEA  SURFACE  AND  WAKE 


NEW  YORK  -  LAGUARDIA  (TWILIGHT) 


MINNEAPOLIS  -  ST.  PAUL  KC  135  TANKER 

INTERNATIONAL  (TWILIGHT) 


MINNEAPOLIS  -  ST.  PAUL  GROUND  TARGETS 

INTERNATIONAL  (NIGHT) 


FIGURE  5.  TYPICAL  STANDARD  VITAL  IV  IMAGERY 


TA8LE  1.  KEY  ISSUES  TO  8E  TRAINED 


A.  Factors  Affecting  Probability  of  Kill  (PK) 

1.  Energy  Management 

a.  own 

b.  threat  energy  state 

2.  Offensive  Weapons  Systems 

a.  switchology 

b.  knowledge  of  best  system  selection 

3.  Assessment  of  Threat  (Current) 

a.  status  assessment 

b.  knowledge  of  what  to  do  about  it. 

B.  Factors  affecting  Probability  of  Survival  (P^) 

1.  Energy  Management 

a.  own 

b.  threat  (know  energy  state  of  threat) 

2.  Defensive  Systems  Management 

a.  display  threats 

b.  respond  to  threats 

3.  Assessment  of  Threats 

a.  status/number 

b.  knowledge  of  what  to  do  about  it. 

C.  Maximizing  PK  x  P<. 

TABLE  2.  GENERIC  TEACHING  AID  SUMMARY 

1.  Make  visible  something  the  pilot  normally 
cannot  see  but  tries  to  model  or  visualize  in  his 
mind. 

2.  Perspective  changes.  Allow  pilot  to  se¬ 
lect  various  viewpoints  in  his  out  of  the  cockpit 
scene  (e.g.,  overview,  threat's  view,  etc.). 

3.  Demonstration  and/or  exaggeration. 

4.  Awareness  stimulators. 

5.  Pointers  for  instructor  and/or  student. 

6.  Sense  exercise. 

7.  Dynamic  student  control  of  scene  and  cues. 

8.  Analogies. 

9.  Cue  indicators. 

10.  Cue  supplementation  or  augmentation. 

11.  Adaptive  aids. 

12.  Scoring  and  immediate  error  feedback. 

13.  Feed  forward  predictors. 

14.  Time  compression  or  expansion. 

15.  Time  quantization  or  task  segmentation. 

16.  Discrete  indicators  (situation  recogni¬ 
tion/response). 

17.  Abstraction 

1.  Make  "visible"  something  the  pilot  nor¬ 
mally  cannot  see,  but  tries  to  model  in  his  mind  - 
For  example,  let's  say  a  pilot  knows  that  another 
aircraft  will  have  to  get  him  within  a  10-degree 
cone  and  within  a  six-mile  range  to  lock  on  some 
particular  kind  of  missile.  This  "cone  of  danger" 
emanating  from  the  nose  of  the  other  aircraft  needs 
to  be  visualized  by  the  pilot,  so  that  he  may  avoid 
it.  This  visualization  in  three  dimensions  under  a 
wide  variety  of  circumstances  could  be  taught  by 
simply  showing  the  cone  emanating  from  the  other 


aircraft  in  a  computer  generated  image  visual  sys¬ 
tem.  As  the  pilot  flies  against  this  target,  he 
can  learn  to  internalize  the  image  of  that  lethal 
cone  for  use  in  the  'real  world',  where  the  cone  is 
invisible.  Obviously,  there  are  many  other  exam¬ 
ples  of  situations  where  this  technique  of  making  a 
cue  available  in  the  simulator  that  is  hidden  in 
the  real  world  could  be  expected  to  be  useful  for 
training. 

2.  Perspective  Changes  -  Advanced  pilots 
generally  do  not  view  their  actions  only  from  their 
own  viewpoint,  but  abstract  the  situation  to  an 
overview  or  "God's  eye  view".  Some  simulator  in¬ 
structor  stations  show  this  viewpoint  to  the  in¬ 
structor  or  to  the  student  after  an  engagement. 
More  immediate  feedback  to  facilitate  this  perspec¬ 
tive  abstraction  could  be  given  by  allowing  the 
student  to  change  the  viewpoint  being  displayed 
while  still  in  the  cockpit.  Other  viewpoints  could 
also  be  helpful,  such  as  how  he  looks  to  a  ground 
threat  or  to  his  opponent.  This  type  of  technique 
might  aid  in  the  development  of  situation  aware¬ 
ness. 

3.  Demonstration/Exaggeration  -  If  a  subtle 
cue  must  be  detected  by  the  pilot,  it  can  be  help¬ 
ful  to  exaggerate  the  cue  first,  so  that  the  pilot 
has  a  good  awareness  of  what  he  is  looking  for.  For 
example,  a  pilot  has  an  altitude  below  which  he  is 
so  involved  in  avoiding  ground  impact  that  he  can 
not  perform  other  tasks.  That  altitude  is  his 
"comfort  level".  If  it  is  desired  to  demonstrate 
to  the  pilot  that  his  comfort  level  depends  on  his 
speed,  a  simulated  course  may  be  flown  at  Mach  3 
and  then  at  30  knots  before  allowing  him  to  learn 
his  comfort  level  for  more  normal  speeds. 

4.  Awareness  Stimulators  -  There  is  a  dan¬ 
gerous  tendency  for  a  pilot  while  performing  a  dif¬ 
ficult  task,  such  as  air-to-ground  or  air-to-air 
attack,  to  focus  on  that  task  to  the  exclusion  of 
perception  of  other  events  around  him.  To  maintain 
the  pilot's  awareness,  other  objects  or  events 
could  be  presented  in  the  scene,  such  as  other  air¬ 
craft,  and  the  pilot's  ability  to  monitor  them  dur¬ 
ing  his  task  could  be  included  in  his  score  for  the 
task. 

5.  Pointers  for  Instructor  and/or  Student  - 
It  has  long  been  known  that  a  simple  pointer  is 

useful  in  communication  about  visual  displays,  yet 
no  aircraft  simulator  has  such  a  pointer  available 
for  use  in  the  out-of-the  window  scene.  There  are 
at  least  three  ways  such  a  pointer  could  be  used 
once  positioned  by  either  the  instructor  or  stu¬ 
dent.  When  released  it  could  remain  stationary  in 
ground  coordinates  (three  dimensional),  or  in  dis¬ 
play  screen  coordinates  (two-dimensional),  or  it 
could  remain  a  fixed  number  of  degrees  below  the 
horizon.  This  last  method  of  operation  would  be 
applicable  to  designating  a  particular  dive  angle 
or  glide  slope. 

6.  Sense  Exercise  -  A  computer  generated 
visual  system  can  readi )y  be  used  to  give  a  student 
practice  in  certain  types  of  fine  perceptual  tasks 
with  feedback  such  as  immediate  knowledge  of  re¬ 
sults.  There  are  many  useful  examples  of  this 
technique;  one  could  practice  closure  rate  judge¬ 
ments,  target  aspect  from  motion  judgements,  land¬ 
ing  flow  pattern  discrimination,  low  contrast  tar¬ 
get  detection,  scan  patterns,  and  so  on. 
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7.  Dynamic  Observer  Control  of  Scene  and 
Cues  -  For  example,  if  it  is  desired  to  teach  a 
student  to  judge  target  aspect,  he  could  be  given 
control  of  the  target  as  if  it  were  a  remotely 
piloted  vehicle.  If  the  aspect  were  uncertain,  he 
could  test  it  by  seeing  how  it  responds  to  his 
control  inputs. 

8.  Analogies  -  It  is  often  useful  to  show 
similarities  between  a  task  to  be  learned  and  a 
more  familiar  task.  It  may  even  be  useful  to  teach 
a  simple  task,  for  later  use  in  analogy  to  a  more 
complex  task. 

9.  Cue  Indicators  -  There  are  many  subtle 
cues  in  flying.  These  may  be  indicated  to  the 
student  in  a  variety  of  ways  such  as,  use  of  a 
pointer,  exaggeration,  elimination  of  other  ex¬ 
traneous  cues,  and  so  forth. 

10.  Cue  Supplements  or  Cue  Augmentation  - 
Subtle  cues  may  initially  be  supplemented  with 

more  obvious  ones,  to  lead  quickly  to  correct  per¬ 
formance  and  then  weaned  away  as  the  student's  pro¬ 
ficiency  increases.  This  would  be  particularly 
adapted  to  the  case  where  the  pilot  is  learning  two 
different  tasks  simultaneously,  while  one  is  de¬ 
pendent  upon  the  other.  For  example,  in  landing, 
the  two  tasks  are:  controlling  the  airplane,  and 
perceiving  the  flight  path.  One  cannot  control  the 
airplane  without  perceiving  the  flight  path  and  one 
cannot  test  one's  flight  path  perception  before 
getting  the  airplane  under  control.  This  contra¬ 
diction  can  be  avoided  by  supplementing  the  nor¬ 
mally  subtle  flight  path  perceptual  cues  until  con¬ 
trol  is  learned  and  then  teaching  the  perceptual 
task  qf  detecting  glideslope  deviations  from  subtle 
cues/ 

11.  Adaptive  Aids  -  This  is  a  broad  category 
which  must  be  carefully  used  to  avoid  student  de¬ 
pendence  on  the  aid,  but  can  be  extremely  effective 
in  quickly  making  the  student  capable  of  performing 
correctly.  Ihe  visible  adaptive  glide  slope  of 
Gaven  Lintern^  of  the  University  of  Illinois  is  an 
example. 

12.  Scor inq/Error  Feedback  -  This  is  not  a 
new  technique,  Dut  it  certainly  could  be  used  in 
new  ways.  Computer  generated  image  out-of-the- 
cockpit  displays  provide  the  opportunity  for 
quicker  feedback,  which  is  known  to  produce  quicker 
learning.  An  example  of  this  technique  would  be  to 
superimpose  scoring  data  on  the  pilot’s  outside 
scene  during  a  training  flight  in  the  simulator. 
For  instance,  the  pilot's  probability  of  survival 
could  be  displayed  in  a  corner  of  the  scene  as  a  bar 
graph  ranging  from  zero  to  one  and  if  the  proba¬ 
bility  gets  too  low,  the  computer  could  display  a 
brief  explanation  of  what  he  has  done  wrong  im¬ 
mediately,  while  he  is  still  in  the  cockpit,  in¬ 
stead  of  waiting  until  later.  For  bombing  prac¬ 
tice,  miss  distances  and  aircraft  parameters  at 
time  of  release  could  be  shown  after  each  bombing 
pass. 

13.  Feed  Forward/Predictors  -  An  example  here 
would  be  during  air-to-air  combat.  When  a  pilot  is 
on  the  offensive,  he  could  be  shown  where  his  pre¬ 
sent  course  and  closure  rate  would  cause  him  to 
intercept  the  target's  flight  plane. 

14.  Time  Compression/Expansion  -  Sometimes 
events  occur  too  quickly  for  the  novice  to  appre¬ 


ciate  them  Individually  and  to  think  through  the 
ramifications.  Conversely,  the  more  experienced 
student  may  need  to  "over-learn''  one  task,  to  be 
able  to  perform  other  tasks  simultaneously.  One 
could  slow  the  system  down  for  the  former  and  speed 
it  up  for  the  latter.  This  technique  might  also  be 
used  to  simulate  the  situation  when  one's  internal 
clock  is  running  faster  than  normal. 

15.  Quantization  of  Time  -  A  task  can  be 
broken  into  discrete  sieps  such  as  in  the  task  tax¬ 
onomy  of  Robert  Meyer/  These  may  be  learned  sin¬ 
gly  in  whatever  order  is  best  suited  to  learning. 
Examples  would  be  the  backward  chaining  technique 
of  Hughes/  a  slide  presentation,  or  flash  cards. 

16.  Discrete  Indicators  -  A  simple  discrete 
indicator  can  be  used  to  teach  the  student  to  re¬ 
cognize  and  respond  to  a  particular  situation.  In 
teaching  low  level  flight,  if  it  is  desired  to  stay 
below  250  feet,  a  simple  tone  or  indicator  could  be 
given  when  that  limit  is  reached  or  exceeded. 

17.  Abstraction  -  In  air-to-air  combat  it  is 
useful  to  view  what  is  happening  and  develop  tac¬ 
tics  from  the  point  of  view  of  where  each  aircraft 
is  in  a  relative  performance  diagram,  such  as  a 
plot  of  air  speed  versus  acceleration  (g  -  forces). 
This  is  an  example  of  abstracting  events.  Another 
example  might  be  to  teach  certain  perceptual  tasks 
in  a  scene  not  at  all  resembling  the  "real  world" 
and  then  transfer  the  skill  to  the  real  world  or 
realistic  simulation  situation. 

SPECIFIC  EXAMPLES 

Specific  examples  of  training  techniques  pos¬ 
sible  using  computer  generated  imagery  follow. 
They  do  not  have  a  one-to-one  relationship  with  the 
generic  technique  list.  Most  of  the  specific  exam¬ 
ples  incorporate  several  of  the  generic  techniques 
at  once.  Also,  each  generic  technique  is  repre¬ 
sented  in  several  specific  examples  to  varying  de¬ 
grees. 

1.  For  the  task  of  training  air  combat  maneu¬ 
vering  defensive  tactics,  it  may  be  useful  to  en¬ 
hance  the  defender’s  awareness  of  the  attacker's 
"lethal  cone".  The  "lethal  cone"  might  be  the  at¬ 
tacking  aircraft's  antenna  pattern  within  which  he 
could  lock  on  and  launch  an  air-to-air  missile. 
This  pattern  could  be  represented  by  a  cone  of  ap¬ 
propriate  range  and  angle  visibly  emanating  from 
the  attacking  aircraft.  This  could  be  modeled  red 
on  the  inside  and  green  on  the  outside  so  that  the 
defender  would  immediately  know  when  he  is  in  this 
danger  zone.  The  main  objectives  here  would  be  to 
teach  the  pilot  how  this  normally  Invisible  cone 
looks  in  three  dimensions  from  the  various  posi¬ 
tions  attained  during  an  engagement,  and  to  allow 
him  to  develop  techniques  for  avoiding  it.  One 
would  have  to  take  care  with  such  a  technique  not 
to  overuse  such  an  added  cue  to  the  point  of  de¬ 
pendency.  (See  Figure  1.) 

2.  In  air-to-air  gunnery  using  tracer  bul¬ 
lets,  it  is  difficult  to  judge  where  the  bullet 
passed  closest  to  the  target,  and  hence  the  miss 
distance.  A  simple  way  to  aid  this  judgement  would 
be  to  indicate  to  the  pilot  where  this  point  is. 
This  could  be  accomplished  in  a  number  of  ways. 
Perhaps  the  simplest  is  to  change  the  brightness  of 
the  tracers  once  they  have  passed  the  target  range. 
(See  Figure  6. ) 
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continuously  aware  of  what  is  going  on  around  him, 
where  he  is,  and  what  to  do  about  it.  This  is  not 
easy  to  do  while  performing  difficult  tasks.  Ad¬ 
vanced  pilots  frequently  comment  that  they  do  not 
always  think  of  their  situation  from  their  own 
point  of  view.  Rather  they  may  view  the  situation 
from  an  overview  or  from  some  abstract  viewpoint 
(discussed  further  in  example  number  17).  As  an 
aid  to  teaching  this  situation  awareness,  one  could 
allow  the  pilot  to  literally  see  what  is  happening 
from  several  different  viewpoints  at  his  selection 
either  during  or  immediately  after  a  simulated  en¬ 
gagement.  Useful  viewpoints  would  include,  in  ad¬ 
dition  to  his  own:  an  overview  or  “God's  eye 
view",  an  opposing  aircraft's  view,  and  a  ground 
threat's  view.  There  are  even  situations  where  a 
viewpoint  attached  to  but  outside  of  his  own 
aircraft  would  prove  of  utility.  (See  Figure  3.) 


3.  There  is  a  large  number  of  situations  in 
which  it  is  useful  for  the  pilot  to  know  his  or 
another  aircraft's  flight  path  either  during  or  im¬ 
mediately  after  a  maneuver.  This  can  apply  both  to 
air-to-air  and  air-to-ground  combat.  In  some  cases 
this  type  of  information  is  already  being  used  in 
training.  On  the  Air  Combat  Maneuvering  Range 
(ACMR)  several  pilots  can  fly  instrumented  real 
aircraft  in  an  engagement  and  later  review  on 
ground  displays  what  happened  as  seen  by  an  in¬ 
structor  operator  from  the  ground  during  the  en¬ 
gagement.  This  is  very  useful,  but  unfortunately 
it  requires  delayed  feedback  of  several  hours. 
Some  simulators  present  this  information  to  the 
instructor/operator,  but  it  is  again  unavailable  to 
the  pilot  without  delay.  There  is  no  reason  at  all 
why  this  information  cannot  be  presented  to  the 
pilot  in  the  same  out-of-the-window  display  he  uses 
in  the  simulator  for  training.  This  can  reduce 
feedback  delay  to  zero  or  negligible  times  which¬ 
ever  yields  the  best  training  for  a  given  situa¬ 
tion.  A  simple  way  to  think  of  implementing  this 
is  to  have  the  aircraft  drop  a  trail  of  zero  muzzle 
velocity  tracer  bullets  as  it  flys  along.  This 
visible  trail  could  be  used  in  a  wide  variety  of 
ways.  It  could  be  viewed  during  an  air-to-air  en¬ 
gagement  or  immediately  afterward.  (See  Figure  7.) 


FIGURE  7.  VISIBLE  PATH,  CONTRAILS,  OR  RI880N5 

This  technique  can  usefully  be  joined  with  the 
one  described  in  the  next  example  (number  4). 

4.  It  is  of  paramount  Importance  for  a  combat 
pilot  to  develop  what  is  referred  to  as  “situation 
awareness.”  This  involves  a  constant  appraisal  by 
the  pilot  of  the  factors  in  Table  1.  He  must  be 


5.  A  simple  method  for  an  instructor  to  enter 
and  store  a  flight  path  into  the  visual  database  to 
be  either  viewed  and/or  followed  by  the  student 
would  be  to  have  him  fly  the  route  or  maneuver 
while  leaving  the  visible  path  as  described  in  ex¬ 
ample  number  3.  The  instructor  could,  for  in¬ 
stance,  lay  a  trail  showing  an  ideal  air-to-ground 
delivery  or  air-to-air  maneuver.  The  student  could 
then  attempt  to  retrace  this  path  while  also  drop¬ 
ping  “bread  crumbs"  to  mark  his  path.  The  two 
paths  could  then  immediately  be  visually  compared 
while  the  instructor  provides  comments.  Alterna¬ 
tively,  the  instructor's  path  could  be  kept  invis¬ 
ible  until  the  student's  maneuver  also  was  com¬ 
pleted  and  then  the  paths  could  be  compared.  (See 
Figure  8.) 


FIGURE  8.  VISIBLE  INSTRUCTOR  DEMONSTRATION  PATH 

6.  Situation  awareness  and  target  aircraft 
motion  judgement  training  could  be  addressed  by  en¬ 
tering  a  large  number  of  moving  airborne  objects  in 
the  scene.  For  this  purpose  they  need  not  be 
models  of  aircraft  but  could  be  any  identifiable 
object.  While  performing  some  task,  for  example 
ordnance  delivery,  the  pilot  would  have  to  monitor 
continually  the  position,  velocity,  and  direction 
of  each  of  the  moving  objects.  These  objects  are 
illustrated  below  as  flying  numbers.  The  pilot 
could  be  tasked  with  calling  out  which  number  is 
closest  or  approaching  the  fastest  and  so  forth. 
If  the  computer  monitored  the  pilot’s  head  and/or 
eye  position.  It  could  even  perversely  bring  ob¬ 
jects  into  view  from  wherever  the  student  was  not 
looking.  This  would  force  him  to  learn  effective 
scan  patterns.  (See  Figure  9.) 
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FIGURE  9.  SITUATION  AWARENESS  TRAINING 


7.  A  cursor  has  long  been  used  as  a  valuable 
tool  in  computer-aided  design  applications.  It 
would  be  equally  valuable  as  part  of  the  computer¬ 
generated  scene  in  aircraft  simulation  visual 
systems.  In  that  application  a  number  of  useful 
modes  of  operation  can  be  envisioned  that  would 
allow  instructor  and  student  to  more  easily 
comnunicate  about  the  visual  scene.  One  mode  would 
be  as  presently  used  in  other  appl ications,  that  is 
a  two-dimensional  cursor  in  display  window 
coordinates.  Another  would  be  a  cursor  that  moved 
in  three-dimensional  ground  coordinates.  A 
particularly  unusual  mode  of  use  in  flying  training 
would  be  a  cursor  that  moved  in  coordinates 
relative  to  the  horizon.  For  example,  the  cursor 
in  this  last  mode  could  be  set  to  30  degrees  below 
the  horizon  and  would  thus  help  the  student  learn 
to  establish  a  30  degree  dive  bombing  angle  to  a 
particular  target.  A  similar  cursor  set  at  3 
degrees  below  the  horizon  would  continuously  point 
to  the  landing  spot  that  would  be  impacted  if  the 
plane  held  a  3  degree  glideslope.  Simply  setting 
the  cursor  to  various  known  degree  settings  below 
the  horizon  would  aid  the  student  In  learning  some 
important  visual  judgements. 

8.  Visual  exercise  may  improve  judgement  of 
air  target  or  other  motion  cues.  The  student  could 
spend  time  looking  at  displays  showing  moving 
objects  (using  monocular  and/or  binocular  cues). 
Such  displays  are  particularly  amenable  to  computer 
generation.  They  need  not  necessarily  look  like 
real  world  objects.  In  fact  the  optimization  of 
what  display  to  show  and  the  feedback  of  results 
schedule  would  be  an  intriguing  study  In  itself. 
It  might  turn  out  that  abstract  displays  of  some 
sort  optimally  exercise  these  visual  senses.  Per¬ 


haps  sense  exercise  is  separable  from  teaching 
judgements  using  that  sense.  The  first  might  ad¬ 
dress  lowering  of  perceptual  thresholds,  while  the 
other  would  address  quantitative  suprathreshold 
judgements.  (See  Figure  14.) 

9.  In  learning  to  judge  distant  target  aspect 
angle  and  motion  it  may  be  very  useful  to  let  the 
student  practice  flying  simulated  remotely  piloted 
vehicles  (RPV's).  This  task  requires  knowledge  of 
aspect  and  motion  to  succeed  and  could  aid  in 
learning  aspect  since  the  aspect  can  be  determined 
from  reactions  of  the  RPV  to  the  pilot's  controls. 
This  task  might  also  help  develop  the  ability  to 
visualize  a  situation  from  another  viewpoint  as 
discussed  in  example  4.  (See  Figure  10.) 

10.  When  a  pilot  is  performing  a  task  that  is 
very  demanding  his  attention  tends  to  concentrate 
on  that  task.  Hi s  attention  to  peripheral  cues  is 
reduced.  There  are  two  ways  to  attack  this  pro¬ 
blem.  One  mentioned  in  example  6  is  to  attempt  to 
teach  him  to  overcome  this  tendency  to  overconcen- 
trate  his  attention.  The  other,  which  mpy  comple¬ 
ment  the  first,  is  to  help  him  learn  to  at  least  get 
the  most  from  the  information  available  in  the  nar¬ 
row  area  of  concentration.  Thus  one  could  practice 
flying  with  minimal  cues  such  as  available  in  the 
immediate  target  area.  (See  Figures  11  and  12.) 

11.  While  flying,  a  pilot  generates  and  main¬ 
tains  an  internal  model  of  where  he  is  with  respect 
to  other  objects  such  as  the  ground,  other  vehi¬ 
cles,  or  a  particular  target.  He  continually  up¬ 
dates  this  model  from  all  available  sources  of  in¬ 
formation  using  all  of  his  senses.  The  ability  to 
accurately  update  this  model  from  limited  or  con 


FIGURE  10.  PRACTICING  ASPECT  ANO  MOTION  JUDGEMENT  BY  FLYING  SIMULATED  REMOTELY  PILOTED  VEHICLES 
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FIGURE  11.  MINIMAL  CUES  -  LEARNING  TO  POSITION  ONESELF  WITH  RESPECT  TO  A  SIMPLE  BALL  OR  CUBE  WITH  NO  OTHER 
CUES  AND/OR  WITH  A  HORIZON 


FIGURE  12.  TARGET  AREA  ONLY  CUES  SCENE 


flicting  information  is  useful.  An  example  would 
be  the  situation  in  which  a  pilot  is  attempting  to 
attack  a  ground  target  by  dive  bombing.  During  the 
initial  attack  phase  the  target  might  appear  off  to 
one  side.  He  could  fly  along  till  the  target 
approaches  90°  off  to  one  side  then  roll  the 
aircraft  to  turn  towards  the  target.  Prior  to  the 
roll  maneuver  he  would  generally  look  sometimes  at 
the  target  to  update  his  internal  model  of  its 
location  with  respect  to  him,  and  at  other  times  in 
front  of  his  aircraft  to  check  his  flight 
direction.  Flying  an  aircraft  in  one  direction 
while  looking  for  a  prolonged  period  in  another 
direction  could  cause  disorientation.  Hence  the 
pilot  is  tempted  to  move  his  eyes  and,  hopefully 
less  so,  his  head  back  and  forth  between  these  two 
data  sources  to  maintain  the  accuracy  of  his 
internal  situational  model.  It  might  be  worthwhi le 
to  give  the  pilot  practice  flying  the  aircraft  in 
one  direction  while  his  field  of  view  is  limited  to 
another  direction  such  as  at  the  target.  He  might 
then  learn  to  update  his  internal  model  accurately 
from  different  information  than  he  usually  uses, 
requiring  reduced  head  movement  for  this  task  and 
allowing  more  concentration  on  the  target.  A 
caveat  is  that  he  still  would  have  to  look  in  other 
directions  to  maintain  awareness  of  other  vehicles 
in  the  area. 

12.  The  ideal  position,  size,  and  shape  of  a 
target  ("sight  picture")  for  a  particular  type  of 
air-to-ground  attack  could  be  shown  for  comparison 
to  how  it  looks  during  the  student's  approach. 
(See  Figure  13.) 


13.  The  angle  of  attack  (the  angle  at  which 

the  airplane  wing  meets  the  air5.)  of  an  airpla  i 
in  flight  is  of  paramount  importance  to  flyin.,. 
Yet,  it  is  not  readily  apparent  in  a  real  airplane 
unless  indicated  by  an  instrument.  A  pilot  must 
learn  to  appreciate  angle  of  attack,  but  in  the 
real  world  other  cues  such  as  those  to  pitch  angle 
are  much  more  prominent  and  at  first  divert  his 
attention  from  this  important  concept.  In  the  com¬ 
puter  generated  simulation  one  can  remove  the  ex¬ 
traneous  cues  and  initially  supplement  the  cues  to 
angle  of  attack.  The  student  could  fly  in  a  scene 
with  no  ground  or  horizon  references  at  all  to 
avoid  confusion  and  the  air  could  be  made  visible 
by  filling  it  with  points  so  that  angle  of  attack 
could  easily  be  judged.  Thus  the  student  would 
first  learn  to  fly  with  respect  to  the  air,  not  the 
ground.  One  could  also,  in  this  environment,  posi¬ 
tion  the  pilot  outside  the  plane  (either  fixed  with 
the  plane  or  fixed  with  the  air)  to  make  visualiza¬ 
tion  and  understanding  of  relative  wind  easier. 
Then,  with  this  firm  foundation,  the  ground  refer¬ 
ence  cues  such  as  pitch  angle  could  be  brought  in 
with  less  confusion.  He  would  for  Instance  become 

more  readily  aware  that  the  stick  is  not  the  up 
lever.  The  scene  for  this  training  situation  might 
have  the  points  representing  the  air  randomly  dis¬ 
tributed  in  a  volume  to  preclude  any  pitch  cues.  A 
similar  scene,  but  with  the  points  regularly  dis¬ 
tributed,  could  be  used  to  teach  perspective  judge¬ 
ments  and  motion  judgements  frequently  used  in  fly¬ 
ing  such  as  location  of  the  "aim  point",  the  point 
toward  which  the  aircraft  is  headed.  Another  use 
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FIGURE  13.  IDEAL  SIGHT-PICTURE  DISPLAY  FOR  COMPARISON  DURING  AIR-TO-GROUND  ATTACK  TRAINING 


for  such  scenes  would  be  to  exercise  visual 
perception  of  motion.  (See  Figure  14.) 

14.  To  aid  in  learning  the  proper  sight 
picture  and  how  to  make  corrections  during  dive 
bombing  one  could  show  a  cone  emanating  from  the 
target  at  the  proper  dive  angle  so  that  the  pilot 
knows  when  he  is  at  the  proper  dive  angle  or  how  far 
off  he  is. 

15.  Although  it  would  be  difficult  to  compute, 
it  could  be  useful  to  provide  the  student  pilot 
with  a  cue  to  his  fluctuating  probability  of 
survival  (P  ),  probability  of  killing  a  target  (P.) 
or  a  combination  of  the  two  such  as  P  x  P.  .  This 
could  be  done  by  showing  an  object  in  a  corner  of 
the  display  which  would  change  as  P^  x  P  ,  P  ,  or  P. 
change.  This  would  be  particularly5  useful  ill 
teaching  tradeoffs  between  offensive  and  defensive 
tactics  in  crowded  dogfights. 

Another  way  to  implement  this  might  be  to  have 
the  display  grow  more  red  as  the  probability  of 
survival  is  reduced.  (See  Figure  15.) 

16.  Showing  the  student  pilot  a  predicted  path 
for  himself  and  his  opponent  including  an 
indication  of  relative  speed  at  intercept  to  help 
him  learn  to  judge  when  he  would  overshoot,  when  to 
bleef  off  energy,  and  when  to  discontinue  bleeding 
off  energy  could  help  give  the  student  a  feel  for 
optimum  solutions  in  various  basic  air  combat 
maneuvering  (ACM)  situations  and  help  him  learn 
energy  management. 


17.  In  air  combat  it  would  be  very  useful  for 
the  student  to  1 earn  to  detect,  take  advantage  of, 
and  affect  a  threat  aircraft's  "g"  state,  energy 
state,  and  where  the  two  aircraft  are  in  their  re¬ 
lative  performance  envelopes.  This  type  of  infor¬ 
mation  could  be  presented  in  the  training  simulator 
even  though  it  might  clog  his  already  overloaded 
sensory  channels  J  put  it  in  the  actual  aircraft 
in  combat.  An  ideal  example  of  this  approach  is 
given  in  Moroney,  et  at0.  (See  Figure  2.) 

DEMONSTRATIONS 

Several  of  the  specific  techniques  have  been 
implemented  on  the  MDEC  VITAI  IV  system.  These 
are: 

1.  -  Visible  lethal  cone 

3.  -  Visible  aircraft  paths  in  sky  (wing 

tip  ribbons). 

4.  -  Multiple  viewpoints. 

7.  -  Cursor 

10.  -  A  minimal  target  cues  scene  in  which 
most  of  the  scene  elements  (horizon, 
target,  ground  grid)  can  be  indepen¬ 
dently  turned  on  or  off. 

13.  -  A  scene  having  no  ground  or  horizon 
references  composed  of  a  solid  cube 
of  light  points. 

A  small  number  of  people  have  flown  these 
scenes  and  found  them  exciting  and  challenging.  It 
will  take  some  practice  utilizing  these  scenes  lu 
learn  what  effects  they  have  and  how  to  make  the 
most  of  them. 
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FIGURE  15.  PROBABILITY  OF  SURVIVAL  AND  KILL  INDICATORS 
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CONCLUSIONS  &  DIRECTIONS  FOR  FUTURE  WORK 

This  report  represents  a  mere  tip  of  the  ice¬ 
berg  compared  to  the  vast  array  of  training  tech¬ 
niques  that  are  made  possible  by  computer  generated 
visual  simulation.  The  objective  of  this  study  was 
to  generate  and  demonstrate  concepts  in  aircrew 
training  methods  that  take  advantage  of  the  flex¬ 
ibility  of  computer  generated  imagery.  While  in 
the  past  the  main  objective  of  simulation  was  to 
duplicate  the  real  world  cockpit  environment,  we 
are  now  in  a  position  to  consider  other  objectives 
that  will  be  attainable  with  training  devices. 
This  requires  a  change  in  orientation  from  thinking 
of  a  simulator  as  an  airplane  substitute,  to  think¬ 
ing  of  it  as  a  training  device  that  can  complement 
"real  world"  training  as  part  of  a  total  training 
system  which  cohesively  runs  the  gamut  from  text 
through  actual  aircraft.  Another  change  requiring 
new  thinking  is  the  expected  change  in  military  use 
of  simulators  from  teaching  only  initial,  simple 
flight  skills  and  procedures  to  teaching  and  main¬ 
taining  of  complex  combat  skills  involving  interac¬ 
tions  among  several  aircraft  and  ground  systems. 

The  study  attempted  to  take  a  step  in  this 
direction  by  creating  examples  of  ways  a  computer 
generated  visual  system  could  be  used  within  this 
context.  Scene  elements  were  incorporated  into  the 
visual  presentation  which  did  not  represent  "real 
world”  objects,  but  which  were  there  solely  for 
instructional  purposes. 

Lists  of  generic  and  specific  training  tech¬ 
niques  were  generated,  some  of  which  are  presented 
here.  Selected  examples  were  implemented  on  a 
VITAL  system  at  McDonnell  Douglas  Electronics  Com¬ 
pany.  A  list  of  key  issues  to  be  trained  was  drawn 
up  to  provide  a  framework  for  concept  generation. 
Upon  examination  of  the  above  mentioned  lists,  it 
was  noticed  that  most  of  the  listed  techniques  are 
intended  to  address  the  teaching  of  "recognitional" 
type  tasks  as  defined  by  Klein1.  They  tend  to  make 
use  of  the  learning  theory  principle  of  immediate 
feedback  of  results  and  are  amenable  to  inclusion 
within  holistic  and/or  adaptive  training  schemes  as 
well  as  more  standard  techniques. 

The  next  stage  of  this  work  will  involve  ex¬ 
ploratory  testing  of  some  of  tne  techniques  dis¬ 
cussed  In  this  report.  A  portion  of  this  work  will 
be  performed  at  MDEC  using  relatively  inexperienced 
pilots  drawn  from  the  flight  student  population  at 
Parks  College  of  St.  Louis  University.  The  class 
of  289  students  should  provide  an  ample  subject 
pool.  The  remainder  of  the  exploratory  testing 
will  be  performed  on  a  VITAL  equipped  flight 
simulator  being  used  by  the  Air  National  Guard 
which  will  provide  a  high  experience  level  subject 
population.  Feedback  from  these  activities  will 
refine  the  present  concepts  and  perhaps  stimulate 
new  ones.  The  results  will  serve  several  pur¬ 


poses.  Feedback  will  be  provided  from  operational 
users  into  the  idea  generation  process  to  more 
directly  and  effectively  serve  their  needs.  The 
study  will  aid  in  the  selection  of  effective  tech¬ 
niques  for  further  more  formal  testing  by  others 
which,  in  turn,  would  provide  the  type  of  hard  data 
required  to  support  incorporation  of  thete  techni¬ 
ques  into  operational  training  programs. 

The  inspiration,  direction,  moral,  and  finan¬ 
cial  support  of  the  Air  Force  Office  of  Scientific 
Research,  and  in  particular  Major  Jack  Thorpe  and 
Dr.  Genevieve  Haddad,  contract  monitors,  is  grate¬ 
fully  acknowledged. 
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ABSTRACT 

LINK  has  developed  techniques  that  significantly  improve  the  usefulness  and  appearance 
of  visual  scenes  without  expending  an  undue  amount  of  visual  system  capability.  Much  uf 
this  work  used  face  or  object  substitution,  real-time  data  base  manipulation,  and  further 
frame  and  field  logic  to  produce  extremely  realistic  sea  scenes  including  sea  state,  bow 
and  stern  wakes,  and  land  scenes  such  as  smoke,  steam,  rotor  blades,  weapons  effects,  etc. 

These  techniques  will  be  discussed  and  a  short  movie  illustrating  some  of  the  most 
dramatic  effects  will  be  shown. 


INTRODUCTION 

First-time  observers  of  digital  visual 
systems  often  react  somewhat  negatively  to  the 
scene.  No  matter  how  many  edges  are  employed, 
the  scene  does  not  contain  nearly  the  amount  of 
detail  that  is  apparent  in  home  TV.  Even  the 
most  detailed  scenes  appear  sterile  and  life¬ 
less.  To  sum  up  the  comments  of  the  observers, 
the  scenes  are  "cartoonish" . 

Even  at  the  most  complex  detail  level,  a 
computer  generated  scene  is  only  an  abstraction 
of  the  real  world.  No  amount  of  hardware, 
texture,  faster  processors,  etc.  will  alter  this 
fact.  That  such  scenes  are  abstract,  however, 
should  not  produce  such  strong  negative 
reactions,  for  the  eyes  and  the  mind  abstract 
constantly  from  the  real  world,  selecting 
information  from  a  shifting  complex  of  images. 

Interestingly,  it  has  been  universally 
observed  that  simulated  scenes  become  much  more 
realistic  when  a  pilot  is  task-loaded.  At  our 
plant,  where  an  interactive  air-to-air  helicop¬ 
ter  combat  system  is  in  operation,  all  of  the 
military  personnel  who  have  flown  in  combat 
have  remarked  that  there  is  no  scintillation, 
data  base  error,  truncation,  nor  any  cartoon¬ 
ish  quality  to  the  display  as  soon  as  the 
"enemy"  fires  a  rocket  past  their  windscreen. 

It  would  appear,  then,  that  as  soon  as  some 
action  or  movement  engages  the  viewer,  the 
problem  of  abstraction  is  very  greatly  re¬ 
duced.  With  this  in  mind,  we  decided  to 
substitute  movement  for  detail  as  a  way  to 
make  simulated  scenes  more  realistic.  By 
adding  dynamic  background  objects  we  have 
made  the  scenes  kinetic  instead  of  static, 
giving  "life"  to  the  displays.  The  move¬ 
ment  keeps  the  eyes  and  mind  busy  and  does 
not  permit  them  to  concentrate  on  the  lack 
of  detail;  the  shifting  quality  of  the  scene 
allows  the  brain  to  perform,  apparently,  in 
a  more  normal  way,  selecting  focus  objects 
from  a  variety  of  images.  We  feel  that 
we  have  made  a  significant  advance  in  the 
'state  of  the  art"  and  that  what  we  have  been 


able  to  accomplish  marks  only  the  beginning 
stages  of  a  process  which  may  aid  computer 
simulation  technology  enormously. 

Only  a  few  of  the  many  special  effects  that 
have  been  developed  will  be  discussed  in  this 
paper.  These  are  dynamic  sea  state,  realistic  bow 
and  stern  wake,  ship  motion,  and  semi-opaque 
images  (suspension  bridge  and  rotor  blades). 

We  will  show  in  detail  how  each  effect  was 
generated  and  present  a  movie  with  each  segment, 
shown  first  in  slow  motion  and  then  in  real-time. 
All  of  these  scenes  were  taken  directly  from  the 
LINK  research  simulator  in  the  Sunnyvale 
Laboratory. 

SEA  STATE 

Static  3D  Waves 

The  3D  waves  were  modeled  as  small  mounds. 
These  "mounds"  were  scaled  smaller  than  hills  and 
coded  blue  rather  than  green.  When  the  results  of 
the  first  model  were  displayed,  it  became  apparent 
that  the  problem  with  3D  waves  is  identical  to  the 
problem  with  the  appearance  of  low  hills  in 
nap-of-the-earth  flight.  The  problem  is  that 
there  is  little  difference  in  the  shading  of  the 
faces  that  make  up  the  "mounds".  When  smooth 
shading  is  applied,  any  resemblance  to  a  3D 
feature  and  any  indication  of  shape  is  totally 
washed  out  (See  Figure  1). 


Figure  1  3D  Static  Waves 
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On  military  helicopter  programs,  we 
investigated  many  modeling  techniques  to  alleviate 
this  problem.  Among  these  techniques  were  overly 
accentuating  sun  shading,  outlining  faces  in  3D 
objects,  and  placing  a  checkerboard  pattern  on  3D 
features.  Whereas  these  techniques  can  be  somewhat 
accepted  by  the  viewer  in  terrain  (there  are 
regular  field  patterns  in  many  areas),  they  are 
totally  unacceptable  in  an  ocean  scene. 

Moving  3D  Waves 

Translating  the  3D  pattern  would  not  change 
its  appearance  (over  a  static  3D  pattern)  because 
the  sun  angle  with  respect  to  the  faces  would 
not  change.  Therefore,  several  other  techniques 
were  implemented.  These  techniques  included 
rotating  the  3D  objects,  switching  3D  objects  in 
and  out  of  the  scene  (leveling),  etc.  All  were 
difficult  to  implement  and  did  not  give  promise 
of  providing  a  useful  and  desirable  visual 
effect  even  if  the  implementation  were 
successful . 

2D  Sea  State 

The  first  attempts  to  develop  sea  state 
were  made  utilizing  a  static  2D  pattern. 

Several  different  regular  patterns  were  modeled  and 
displayed.  Although  these  pattern  types  have 
been  tentatively  accepted  for  nap-of-the-earth 
flight,  their  regularity  makes  them  discon¬ 
certing  for  expanses  of  water.  Originally  it 
had  been  assumed  that  the  patterns  could  be 
modeled  with  subtle  differences  in  intensity  so 
that  the  appearance  would  not  be  objectionable 
but  the  eye  is  much  too  sensitive  to  even  slight 
intensity  changes,  particularly  along  straight 
lines  between  faces.  An  example  of  a  non-smooth- 
shaded  regular  pattern  with  many  intensity 
levels  is  shown  in  Figure  2. 


Figure  2  Ocean  Surface 
(Without  Smooth  Shading) 


These  patterns  were  also  modeled  so  that 
they  could  be  smooth  shaded.  That  is,  the 
intensity  between  faces  was  extrapolated  along 
each  display  scanline  to  eliminate  the  sharp 
discontinuity  in  intensity  from  one  face  to  the 
next.  It  was  presumed  that  this  "smoothing"  in 
a  2D  pattern  would  be  sufficient  to  give  an 
adequate  visual  impression  of  ocean  surface. 


The  pattern  of  Figure  2  was  smooth  shaded 
and  is  shown  in  Figure  3.  This  is  obviously  an 
improvement  over  a  non-smooth-shaded  sea  state. 
However,  the  scene  appeared  much  too  static  and 
lifeless  to  properly  convey  an  ocean  surface. 
For  this  reason,  it  was  decided  that  a  method  of 
changing  the  apparent  intensities  of  the  ocean 
patterns  as  a  function  of  time  would  be 
necessary. 


Figure  3  Ocean  Surface 
(With  Smooth  Shading) 


Dynamic  2D  Sea  State 

The  basic  approach  in  generating  a  dynamic 
sea  was  to  model  a  pattern  of  smooth-shaded 
faces  with  modified  vertex  normals.  Since  the 
cross-product  of  the  vertex  normals  and  the  sun 
vector  controls  the  shading  of  the  faces,  many 
different  patterns  can  be  generated  by  modifying 
vertex  normals  while  the  contrast  of  these 
patterns  can  be  increased  by  moving  the  sun 
downward  from  the  zenith.  The  latter  is  shown 
clearly  in  Figures  4,  5,  and  6.  Figure  4  was 
taken  without  a  sea  pattern  and  Figures  5  and  6 
show  the  contrast  of  the  smooth-shaded  sea 
oattern  at  two  different  sun  angles. 


Figure  4  Sea  State  1 
(No  Pattern  Contrast) 


The  initial  implementation  of  this  approach 
was  to  modify  the  vertex  normals  in  the  object 
descriptions  of  the  sea  pattern  in  a  predetermined 
sequence.  This  would  cause  the  faces  to  change  in 
a  cyclic  sequence.  It  was  determined  that  it  was 
necessary  to  update  the  normals  at  display  frame 
rates  to  prevent  noticeable  stepping  between 
patterns. 
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Figure  5  Sea  State  1-? 
(Moderate  Pattern  rontrast) 


Figure  6  Sea  State  2-3 
(High  Pattern  Contrast) 


Updating  the  normals  at  frame  rates  proved 
to  be  an  extremely  difficult  software  task  and 
the  time  necessary  for  its  execution  exceeded 
the  available  Central  Processing  Unit  (CPU)  time. 

A  simpler  method  of  implementation  was  then 
developed  as  follows  (See  Figure  7).  A  single  sun 
vector  is  normally  sent  each  frame  for  all  objects 
in  the  data  base.  By  separating  the  objects  for 
the  sea  pattern,  it  was  possible  to  assign  a 
second  sun  vector  for  this  particular  group  of 
objects.  Computing  only  a  new  sun  vector  each 
frame  for  the  entire  sea  pattern  made  it  possible 
to  achieve  the  same  visual  appearance  as  the 
original  attempt  at  implementation  (updating  each 
of  the  vertex  normals),  but  with  considerably  less 
real-time  software.  The  single  sun  vector 
computation  for  the  sea  state  required  less  CPU 
time  in  the  real-time  software  than  in  the  initial 
vertex  normal  implementation.  Kith  only  one  vector 
to  change,  it  was  also  possible  to  use  a  more 
complex  vector  modification  algorithm. 

The  sun  vector,  when  rotated  in  a  circular 
motion,  causes  the  sea  patterns  to  change 
realistically.  Changing  the  rotation  rate 
simulates  various  agitations  of  ocean  surface. 
Figures  8  and  9  show  how  the  intensity  of  each 
square  in  the  sea  pattern  changes  as  the  sun 
alters  its  position  as  a  function  of  time.  The 
smooth  shading  was  turned  off  for  these  pictures 


Figure  7  Dynamic  Sea  State  Techniques 


Figure  8  Ocean  Surface  Time  =t, 
(Without  Smooth  Shading) 


Figure  9  Ocean  Surface  Time  *t? 
(Without  Smooth  Shading)  £ 
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Figure  10  Ocean  Surface  Time  =t7 
(With  Smooth  Shading) 

to  make  the  technique  obvious.  Figure  10  is  the 
sea  pattern  shown  in  Figure  9  with  the  smooth 
shading  turned  on.  This  effect,  coupled  with  the 
fact  that  the  angle  of  the  sun  vector  controls  the 
contrast  on  the  pattern  (as  described  previously) 
is  useful  in  generating  a  number  of  different  sea 
states. 

Three  different  patterns  of  vertex  normals 
were  programmed  and  displayed: 

1.  When  the  vertex  normals  ■  re  assigned  in  a 
random  basis,  the  resulting  dynamic  pattern 
was  that  of  a  confused  sea. 

2.  When  the  vertex  normals  were  computed  as  a 
sine  wave  on  odd  rows  and  its  mirror  image  on 
even  rows,  the  pattern  looked  unrealistically 
distinct. 

3.  When  the  vertex  normals  were  computed  as  a 
sine  wave  on  all  rows,  the  pattern  appeared 
as  a  series  of  bands  moving  in  parallel,  not 
unlike  ocean  swells. 

Of  the  three  patterns  mentioned,  the  random 
pattern  appeared  to  be  the  most  acceptable 
visually  and  was  incorporated  into  the  NASA  /Ames 
Digital  Visual  Simulator. 

BOW  AND  STERN  WAKE 

Face  Substi tution 

In  an  attempt  to  generate  moving  objects  such 
as  the  ship  bow  wake,  many  different  schemes  were 
considered.  Usually  these  amounted  to  a  wake 
moving  object  which  traveled  with  the  ship. 

Efforts  were  made  to  make  the  complex  bow-wave 
motion  using  the  simplest  real-time  motion 
possible.  Initial  attempts  made  the  moving  object 
in  the  shape  of  a  cam.  As  the  object  rolled  about 
its  axis  (as  it  moved  with  the  ship),  it  would 
continuously  present  a  different  shape  (and  height 
above  water)  to  the  eye.  The  implementation  of 
this  scheme  proved  to  be  quite  laborious  and 
required  two  real-time  moving  objects  (one  for 
each  side  of  the  ship)  involving  many  scene  edges. 


Attempts  to  simplify  the  “moving  object" 
procedure  indicated  that  a  method  would  have  to 
be  developed  that  was  a  radical  departure  from 
the  straight-forward  approach.  The  movie 
industry  animation  methodology  was  employed  and 
a  series  of  sequenced  faces  were  used  to  make  an 
object  enlarge  and  decay  in  a  programmed 
fashion.  A  procedure  was  developed  whereby  all 
the  sequential  faces  were  made  part  of  the  moving 
object  with  which  they  were  associated--in 
our  case,  with  the  ship  moving  through  the  sea. 
This  procedure  is  shown  graphically  in  Figure  11. 
Four  actual  frames  taken  from  the  display  are 
shown  in  Figure  12.  As  the  moving  object  was 
transferred  from  the  active  data  base,  one  of 
the  bow-wake  faces  was  transferred  with  it  and 
subsequently  displayed.  After  a  fixed  number 
of  frames,  a  pointer  would  substitute  a  second 
face  for  the  first  and  subsequently  display  it 
with  the  ship.  The  rate  at  which  the  subsequent 
faces  were  transferred  and  the  difference 
between  subsequent  faces  produced  the  dynamic 
movement.  The  selection  of  these  two  parameters 
along  with  the  selection  of  various  colors  can 
be  used  to  produce  a  wide  range  of  effects. 

This  technique  of  effectively  modifying  the 
digital  data  base  in  real-time  gave  rise  to  many 
of  the  other  effects  discussed  herein. 


Figure  11  Moving/Growing  Objects 
(Bow  and  Wake) 
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MICROCOPY  RL SOLUTION  TEST  CHARI 


Use  of  Modified  Sea  State  Pattern 

The  bow  wake  generated  was  extremely 
realistic  when  viewed  from  an  eye  position  close 
to  the  surface  of  the  sea.  However,  when 
observed  from  near  vertical  the  faces  comprising 
the  bow  wake  become  almost  invisible  (the  faces 
have  no  thickness).  Techniques  to  make  the  bow 
wake  a  30  pattern  were  complex.  It  was  decided 
that  the  design  approach  used  to  implement  the 
sea  state  pattern  could  also  be  used  for  bow  and 
stern  wake.  Several  patterns,  pattern  colors, 
pattern  sizes,  sun  vector  angles,  and  sun  vector 
speeds  were  modeled  and  displayed  on  the 
research  simulator.  After  several  iterations 
when  the  speed  of  the  sun  vector  rotation  was 
increased  substantially,  a  "churned  water" 
effect  was  created.  Figure  13  shows  the  bow  and 
stern  wake  taken  from  the  research  simulator. 


a  substantial  amount  of  ship  pitch  and  roll. 
Other  ship  movements,  such  as  heave  and  sway,  are 
important  to  landing/takeoff  simulation  but 
do  not  make  additional  demands  on  the  visual 
system.  Discussions  with  Navy  personnel 
established  that  landing  and  takeoff  can  reasonably 
be  expected  to  be  performed  with  pitch  less 
than  5"  and  roll  less  than  8°. 

The  worst  case  for  ship/sea  occulting  would 
be  from  a  position  close  to  the  ocean  surface. 
Viewed  from  this  position,  it  appeared  necessary 
to  model  to  some  distance  below  the  waterline  so 
that  the  hull  can  be  seen  when  the  ship  is  pitched 
up,  and  conversely,  less  hull  above  the  waterline 
can  be  seen  when  the  ship  is  pitched  down. 

This  causes  a  mutually  exclusive  visual  object 
priority  problem.  If  the  ship  has  higher 
priority  than  the  ocean,  the  ship  can  be  seen 


below  the  waterline  to  the  point  that  It  was 
modeled  (this  makes  It  appear  to  “float”  on  the 
ocean  at  the  lowest  point  digitized).  On  the 
other  hand,  if  the  ocean  were  to  have  higher 
priority  than  the  ship,  the  ship  would  not  appear 
except  for  the  area  above  the  horizon.  Any 
occulting  schemes  that  involved  paths  or  seams  in 
the  ocean  were  discarded  because  they  would  permit 
only  preprogrammed  ship  motion.  The  following  is 
a  discussion  of  the  experimental  methods. 


"Moving  Box"  Occulting  Method.  Occulting  the 
part  of  the  ship  which  is  Below  the  water  can  be 
accomplished  by  positioning  a  box  (open  at  the 
top)  around  the  ship  from  the  waterline  down.  The 
box  must  be  the  same  color  as  the  surrounding 
water  and  must  have  a  higher  priority  than  the 
ship. 

The  assignment  of  color,  intensity  and 
priority  of  the  box  are  easily  accomplished. 

Color  and  intensity  are  chosen  during  modeling. 

The  priority  of  the  box  is  accomplished  by  making 
the  box  a  separate  object  list  and  setting  its 
priority  during  initialization  of  the  real-time 
program  to  be  greater  (lower  number)  than  the 
ship's  parts. 

The  accuracy  of  this  method  decreases  with 
increasing  altitude  of  the  viewing  point.  This 
accuracy  can  be  maximized  by  making  this  box  as 
close  to  the  shape  of  the  occulted  parts  of  the 
ship  as  possible.  The  ship  then  appears  to  rise 
from  and  disappear  into  the  sea  when  it  actually 
is  being  occulted  by  the  moving  box. 

This  method  was  implemented  on  the  research 
simulator  and  is  extremely  effective.  However,  a 
simpler  method  (using  no  moving  object  and  fewer 
edges)  is  more  desirable. 

Face  Substitution  Method.  It  was  originally 
assumed  that  if  ships  were  to  be  modeled  only  to 
the  waterline  (effectively  "floating"  on  the  ocean 
surface),  an  eyepoint  close  to  the  water  would 
detect  space  below  the  bow  and  stern  when  the  ship 
pitched  up  and  down.  Before  the  modeling  of  a 
ship  was  completed,  several  methods  of  occulting 
other  than  the  moving  object  method  discussed  in 
the  previous  paragraph  were  investigated.  The 
most  promising  of  these  was  a  modification  of  the 
face  substitution  method  used  -,or  the  bow  wake. 
Basically,  it  involved  using  a  series  of 
wedge-shaped  faces  of  progressively  larger  sizes 
to  "fill"  the  space  between  the  bottom  edge  of  the 
ship  (waterline)  and  the  surface  of  the  ocean  as 
the  ship  pitched  up,  and  the  removal  of  the 
wedge-shaped  faces  as  the  ship  pitched  down.  These 
would  be  inserted  (or  removed)  in  real-time  as  a 
function  of  the  angle  of  pitch  (See  Figure  14). 

(Roll  is  not  a  problem  because  the  face  used 
on  the  bottom  of  the  ship  appears  to  be  simply  the 
extension  of  that  portion  of  the  ship  below  the 
waterline  exposed  during  roll.) 
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Figure  14  Face  Substitution 
Necessity  of  Occulting 

When  the  modeling  of  a  Landing  Platform 
Helicopter  (LPH)  was  completed,  it  was  programmed 
as  a  real-time  moving  object  and  driven  realis¬ 
tically  through  the  ocean  data  base.  Pitch  was 
programmed  in  as  10°  which  is  double  the  maximum 
worst  case  excursion  permissible  in 
landing/takeoff.  The  LPH  in  these  experiments  was 
modeled  only  to  the  waterline.  The  completed 
scene  was  dynamically  demonstrated  to  numerous 
pilots.  At  no  time  during  any  of  these 
demonstrations  was  any  objection  raised,  nor  was 
attention  called  to  the  fact  that  when  the  ships 
pitched  down,  the  length  of  the  prow  did  not 
shorten.  In  fact,  the  comments  were  unanimous 
that  the  ships  motion  was  "realistic".  The  movie 
that  accompanies  this  paper  bears  out  this 
contention. 

SEMI -OPAQUE  IMAGES 
Suspension  Bridge  and  Rotor  Blades 

A  technique  of  substituting  visual  scene 
faces  in  real-time  was  demonstrated  as  being 
feasible  and  useful  in  the  creation  of  the  bow 
wake  previously  discussed.  The  technique  provided 
a  means  of  generating  special  effects  that 
achieved  a  high  degree  of  realism. 

The  basis  of  the  face  substitution  procedure 
was  the  concept  of  manipulating  the  visual  data 
base  in  real-time.  Heretofore,  the  data  base  was 
produced  off-line  and  simply  called  up  when 
required  as  various  levels  of  detail  (usually  as  a 
function  of  range)  or  as  an  entire  moving  object 
(aircraft,  ship,  etc.).  The  face  substitution 
technique  showed  how  effects  that  required  rapid 
image  movement  could  be  produced  easily. 

The  technique  was  further  expanded  to  include 
object  substitution  as  well  as  substitution  of 
objects/faces  on  alternate  fields  or  frames  to 
produce  a  semi -opaque  effect.  The  semi -opaque 
effect  comes  about  by  displaying  the  entire  scene 
on  one  field  or  frame  and  the  object  desired  to 
appear  semi-opaque  on  the  next  field  or  frame. 
Since  the  priority  of  the  semi-opaque  object  would 
obscure  the  scene  behind  the  object  only  every 
other  frame,  the  eye  would  alternately  see  both 
(See  Figure  15).  The  effect  is  the  appearance  of 
a  semi-opaque  object.  The  following  describes  how 
the  technique  is  implemented  and  some  of  the 
interesting  effects  produced. 
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HELICOPTER  ROTOR  ILADES 


Figure  15  Semi-Opaque  Objects 

Object  substitution  is  accomplished  by 
modeling  the  data  base  with  a  dummy  object  in 
the  location  of  the  special  effect.  This  dummy 
object  is  not  displayed  but  is  used  for 
generating  planes  so  that  the  special  effect 
will  have  the  correct  priority.  The  desired 
special  effect  is  modeled  as  a  series  of  objects 
in  the  same  manner  as  a  cartoon  is  generated. 

These  objects  are  then  sequentially  displayed  at 
object  substitution  rates  dependent  upon  the 
particular  application. 

The  suspension  bridge  is  modeled  so  that 
during  the  odd  television  display  fields  only 
the  bridge  towers  and  bridge  deck  are  output  to 
the  object  list  (See  Figure  16).  This  permits 
the  scene  behind  the  bridge  to  be  displayed. 

During  the  even  fields  the  towers  and  deck  are 
again  displayed  (hence  appear  totally  opaque) 
and  the  objects  that  make  up  the  sides  of  the 
bridge  are  also  displayed.  This  gives  an 
excellent  Impression  that  the  city  (or  any 
object)  behind  the  sides  of  the  bridge  is  only 
partially  obscured  by  the  bridge  wires. 

An  actual  image  taken  from  the  research 
simulator  display  Is  shown  in  Figure  17.  It 
clearly  shows  how  effectively  this  technique  can 
be  applied. 

There  is  another  factor  Involved  here  which 
may  not  be  obvious  at  first— the  repetition  rate 
of  the  displayed  Image.  In  the  case  of  the 
bridge,  the  image  cannot  appear  to  flicker  unreal¬ 
istically;  hence  it  must  be  displayed  faster  than 
20  times  per  second.  To  accommodate  both  the 
update  rate  requirement  and  the  alternate  Image 
requirement.  It  was  necessary  to  update  the 
digital  visual  system  60  times  per  second. 


Figure  16  Semi-Opaque  Objects 
(Alternate  Field  Technique) 


Figure  17  Suspension  Bridge 


The  second  example  presented  here  is  the 
helicopter  rotor  blades.  The  object  substitution 
technique  is  used  to  provide  a  semi-opaque  quality 
to  the  spinning  rotor  blades.  It  should  be  noted 
that  in  this  case  two  45°  segment  German  crosses 
have  been  modeled  so  that  they  are  rotated  45°  in 
the  data  base  (providing  a  complete  solid  disc) 
and  displayed  on  alternate  frames  (rather  than 
alternate  fields  as  In  the  case  of  the  bridge). 
The  block  diagram  is  shown  in  Figure  18.  This 
provides  the  semi-opaque  effect  as  discussed 
previously.  In  addition,  displaying  each  set  of 
blades  on  alternate  frames  rather  than  alternate 
fields  as  noted  previously  provided  an  unexpected 
effect.  The  effect  of  the  "flicker"  resulting 
from  the  15  image  per  second  update  (each  blade 
pattern  every  other  frame)  gives  a  realistic 
Impression  of  blade  movement  that  was  Impossible 
to  obtain  in  any  other  manner. 


90 


In  this  case,  an  undesirable  attribute  of  a 
discrete  image  system  Is  being  used  to  advantage. 
Figure  19  shows  the  semi-opaque  quality  of  the 
blades,  but  unfortunately  only  direct  viewing  on 
the  display  CRT  or  in  a  movie  can  show  how  this 
simple  technique  provides  realistic  blade 
"motion". 


Figure  18  Semi -Opaque  Objects 
(Alternate  Frame  Technique) 


CONCLUSION 

Two  Important  digital  visual  techniques  have 
been  discussed  in  this  paper.  The  first  was  the 
real-time  data  base  manipulation  procedures  which 
were  used  for  producing  rotor  blades,  bow  wakes, 
and  weapons  effects  without  resorting  to  the  use 
of  moving  objects.  The  second  was  the  separation 
of  the  object  list  in  the  data  base  so  that 
separate  sun  vectors  for  different  sets  of  objects 
could  be  output  to  the  image  processor.  In  the 
latter  case,  the  appearance  of  dynamic  sea  state 
resulted  from  the  use  of  a  constantly  changing  sun 
vector  and  a  randomly  modeled  blue  checkerboard. 
The  results  of  the  application  of  just  these  two 
techniques  are  an  indication  of  the  kinds  of 
visual  effects  that  remain  to  be  Implemented  on 
digital  visual  systems  with  little  more  than 
imagination. 

Both  of  the  techniques  used  can  also  be 
considered  successful  because  little  additional 
data  base  is  required,  few  additional  edges  have 
to  be  processed,  and  only  an  insignificant  amount 
of  real-time  is  required.  In  these  particular 
cases,  it  can  be  stated  that  the  visual  impact  of 
the  resulting  scene  is  far  greater  than  the  small 
amount  of  visual  system  capacity  that  it  uses. 
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Figure  19  Helicopter  Rotor  Blades 
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ABSTRACT 


Three  types  of  visual  scene  cues  were  varied  in  order  to  determine  their  effect  on 
pilot  performance  during  simulated  low  altitude  flight.  The  three  types  of  visual  cues 
consisted  of  three  sizes  of  ground  texture  patterns,  the  presence  or  absence  of  vertical 
object  cues,  and  the  presence  or  absence  of  an  aircraft  shadow.  The  pilots  who  flew  the 
simulated  missions  reported  that  all  three  visual  cues  were  useful,  however  the  vertical 
object  cues  and  texture  patterns  were  more  useful  than  the  aircraft  shadow.  Both  the 
texture  patterns  and  the  vertical  object  cues  produced  statistically  significant  differ¬ 
ences  in  quantitative  measures  of  pilot  performance. 


INTRODUCTION 

The  purpose  of  this  study  was  to  assess 
the  effects  of  three  types  of  visual  scene  cues 
upon  simulated  low  altitude  terrain  flight  mis¬ 
sions  in  A-10  aircraft.  The  three  types  of 
visual  scene  cues  studied  were  ground  texture 
patterns,  vertical  object  cues,  and  the  aircraft 
shadow.  The  lack  of  adequate  visual  scene  tex¬ 
tural  detail  is  often  considered  to  be  a  limiting 
factor  in  the  use  of  computer  generated  imagery 
for  nap-of-the-earth  (NOE)  flight  simulation. 

Low  altitude  terrain  flight,  including  NOE  maneu¬ 
vering  and  contour  flight,  and  aircraft  flare  and 
landing  seem  to  require  visual  scene  textural 
detail  for  optimum  pilot  judgment  of  distance 
above  the  ground.  The  actual  level  of  textural 
detail  required  may  vary  as  a  function  of  air¬ 
craft  speed.  For  example,  it  may  be  possible  to 
use  much  grosser  textural  detail  for  A-10  NOE 
flight  than  for  helicopter  flight.  Even  with  the 
limited  edge  capacity  of  the  current  Advanced 
Simulator  for  Pilot  Training  (ASPT)  system,  it  may 
be  possible  to  greatly  enhance  pilot  performance 
during  NOE  flight  maneuvers,  using  fairly  large 
texture  patterns.  Also,  by  limiting  the  aircraft 
altitude  and  through  the  use  of  rolling  terrain 
it  should  be  possible  to  keep  the  visual  scene 
edge  requirements  within  the  limits  of  the  cur¬ 
rent  ASPT  Computer  Generated  Image  (CGI)  system. 
Thus,  one  goal  of  this  study  was  to  study  the 
effects  of  maximum  visual  texturing,  within  the 
current  ASPT  visual  system,  upon  pilot  perfor¬ 
mance  during  NOE  and  low  altitude  contour 
fl ights. 

Vertical  object  cues  are  also  quite 
important  for  pilots  to  judge  aircraft  height 
above  the  ground.  The  relative  trade-offs 
between  ground  textural  cues  and  vertical  object 
cues  will  be  Important  in  determining  the  optimum 
utilization  of  limited  computer  generated  image 
edge  resources.  The  relative  importance  of 
vertical  object  cues  versus  ground  texture  pat¬ 
terns  will  also  be  useful  in  assessing  the 
utility  of  CGI  hardware  options  which  generate 
synthetic  texture  for  visual  scenes.  Thus,  simu¬ 
lated  missions  were  also  flown  with  and  without 
vertical  object  cues  in  order  to  study  their 
effect  on  pilot  performance. 


The  aircraft  shadow  has  also  been  proposed 
as  an  important  visual  cue  for  low  level  flight. 
Since  this  visual  cue  also  imposes  an  additional 
burden  on  the  CGI  system  it  is  important  to 
assess  its  proven  utility  as  a  visual  cue  for 
terrain  flight.  As  with  the  vertical  object  cues, 
half  of  the  simulated  missions  were  flown  with  and 
half  were  flown  without  the  aircraft  shadow  in 
order  to  study  its  effect  on  pilot  performance. 

METHOD 

The  pilot's  task  consisted  of  flying  mis¬ 
sions  in  a  simulated  A-10  aircraft  from  an 
initialization  point  across  approximately  ten 
miles  of  rolling  terrain,  which  consisted  of  eight 
valleys  separated  by  low  hills  that  were  either 
100  or  300  feet  high.  Twelve  A-10  pilots,  who 
were  qualified  as  either  instructor  pilots  or 
combat  mission  ready  pilots,  flew  in  the  study. 

The  flying  task  primarily  consisted  of  maintaining 
a  very  low  altitude  flight  profile  consisting  of 
both  nap-of-the-earth  flight  around  vertical 
objects  and  contour  flight  which  followed  the 
profile  of  the  terrain.  Low  altitude  flight  was 
imposed  by  instructions  to  the  pilots  and  by  feed¬ 
back  concerning  performance  measurement  scoring. 
Pilot  performance  was  quantitatively  measured  in 
several  ways.  One  primary  measure  consisted  of 
scoring  pilot  ability  to  maintain  aircraft  alti¬ 
tude  at  50  feet  plus  or  minus  30  feet,  while  fly¬ 
ing  in  the  flat  valleys.  This  score  was  computed 
as  a  percentage  of  the  time  within  the  altitude 
tolerance  band  from  20  to  80  feet  above  ground 
level  (AGL).  The  pilots  were  also  instructed  to 
try  to  crest  the  hills  at  50  feet.  Actual  air¬ 
craft  altitude  values  were  collected  at  the  top 
of  each  hill  as  well  as  the  minimum  and  maximum 
altitude  while  flying  over  each  hill  contour. 

The  minimum  altitude  value  attained  while  flying 
in  each  valley  was  also  saved  as  a  data  point. 
Terrain  crashes  (strikes)  were  also  detected  and 
scored  as  the  cumulative  time  spent  in  contact 
with  the  ground.  Airspeed  was  also  controlled  by 
having  each  pilot  fly  the  course  at  300  knots. 

Time  within  tolerance  scoring  was  also  used  for 
the  aircraft  airspeed  with  a  tolerance  band  of 
plus  or  minus  15  knots. 
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Three  types  of  visual  cues,  which  served 
as  the  independent  variables,  were  randomly 
varied  in  order  to  control  for  learning  effects. 
The  visual  cues  consisted  of  three  different 
sizes  of  checkerboard  texture  patterns,  the 
presence  or  absence  of  vertical  objects,  and  the 
presence  or  absence  of  the  A-10  aircraft  shadow. 
There  were  thus  twelve  unique  combinations  of 
visual  cues  which  were  randomly  presented  to  each 
pilot  in  a  unique  random  order.  The  sizes  of  the 
checkerboard  patterns  were  either  220,  440,  or 
880  feet  on  a  side. 

Each  pilot  flew  the  corridor  three  times 
for  each  unique  combination  of  visual  cues,  for  a 
total  of  36  experimental  sorties.  Six  initial 
sorties  were  also  flown  at  the  beginning  of  data 
collection  in  order  to  familiarize  the  pilots  with 
the  mission  profile  and  the  scoring  feedback,  and 
to  reduce  initial  learning  effects. 

RESULTS  AND  CONCLUSIONS 

In  general  the  pilots  who  flew  the  simu¬ 
lated  missions  reported  that  all  three  types  of 
visual  cues  were  useful,  however  the  vertical 
object  cues  and  texture  patterns  were  of  greater 
help  than  the  aircraft  shadow.  Some  pilots 
reported  that  the  aircraft  shadow  was  particularly 
useful  in  signaling  impeding  contact  with  the 
ground.  The  vertical  object  cues,  especially 
trees  of  a  known  height,  were  subjectively  very 
useful  in  gauging  height  above  the  terrain.  The 
texture  patterns  were  also  reported  as  desirable, 
but  perhaps  in  a  less  conscious  fashion  than  the 
vertical  object  cues.  The  pilots  reported  a 
definite  preference  for  the  smallest  texture  pat¬ 
tern,  which  used  squares  that  were  220  feet  on  a 
side  rather  than  the  larger  patterns.  They  also 
especially  disliked  flying  over  the  largest  tex¬ 
ture  pattern  without  vertical  object  cues.  The 
pilots  also  would  have  preferred  more  irregular 
"natural"  patterns  rather  than  the  highly  regular 
checkerboard  patterns. 

The  quantitative  data  from  this  study 
indicate  that  both  the  texture  patterns  and  the 
vertical  object  cues  produced  statistically  sig¬ 
nificant  differences  in  pilot  performance.  These 
Initial  statistical  analyses  were  performed  using 
a  multivariate  analysis  of  variance  (MANOVA)  on 
several  measures  at  once,  with  step  down  analyses 
of  variance  (ANOVA's)  for  the  individual  vari¬ 
ables.  The  MANOVA  probabilities  for  these  two 
variables  were  both  less  than  .001.  However, 
only  the  texture  pattern  cues  produced  a  signifi¬ 
cant  effect  (p<  .001)  on  the  time  within  toler¬ 
ance  scoring  for  altitude  in  the  valleys.  The 


presence  or  absence  of  the  vertical  objects  did 
not  significantly  effect  this  measure.  The 
average  scores  across  pilots  for  the  three  tex¬ 
ture  patterns  were  64.7*  for  the  880  foot,  72.6* 
for  the  440  foot  and  77.4*  for  the  220  foot  tex¬ 
ture  patterns.  The  average  score  without  vertical 
objects  was  70.9*,  and  with  vertical  objects  it 
was  72.2*.  The  texture  patterns  also  produced  the 
only  significant  effect  (p<  .003)  on  the  average 
minimum  altitude  values  in  the  valleys.  These 
average  values  were  47.0  feet  for  the  880  foot 
pattern,  45.5  feet  for  the  440  foot  pattern  and 
43.0  feet  for  the  220  foot  pattern.  The  average 
value  without  vertical  objects  was  45.5  feet  and 
with  vertical  objects  it  was  44.8  feet. 

The  vertical  object  cues  had  a  significant 
effect  (p  <  .001)  on  the  average  aircraft  altitude 
(AGL)  at  the  top  of  the  hills.  The  average  alti¬ 
tude  without  vertical  objects  was  72.7  feet  and 
with  vertical  objects  it  was  63.6  feet.  The  63.6 
foot  average  value  was  closer  to  the  requested  50 
foot  clearance  at  the  hill  tops.  The  texture  pat¬ 
terns  did  not  produce  a  significant  effect  on  the 
average  aircraft  altitude  at  the  top  of  the  hills. 
The  average  values  were  69.2  feet  for  the  880  foot 
pattern,  67.6  feet  for  the  440  foot  pattern  and 

67.8  feet  for  the  220  foot  pattern.  Both  the 
vertical  object  cues  (p<  .001)  and  the  texture 
patterns  (p<  .001)  produced  significant  effects 
on  the  average  minimum  altitude  values  that  occur¬ 
red  over  each  hill.  The  average  minimum  altitude 
values  were  47.9  feet  for  the  880  foot  pattern, 

45.9  feet  for  the  440  foot  pattern  and  40.6  feet 
for  the  220  foot  pattern.  The  average  minimum 
altitude  values  were  48.2  feet  without  the  verti¬ 
cal  objects  and  41.4  feet  with  the  vertical 
objects.  Apparently  the  pilots  flew  closer  to  the 
hill  surface  when  both  the  vertical  objects  and 
the  smallest  texture  patterns  were  present. 

None  of  the  visual  cue  variables  signifi¬ 
cantly  affected  the  amount  of  time  crashed.  In 
general,  this  measure  was  low  with  some  pilots 
crashing  into  the  simulated  terrain  more  fre¬ 
quently  than  other  pilots.  There  were  also  no 
statistically  significant  differences,  due  to  the 
visual  cues,  in  the  aircraft  airspeed  time  within 
tolerance  scores. 

Based  upon  the  initial  data  analyses,  the 
textural  visual  cue  variable  appears  to  have  a 
stronger  effect  on  pilot  performance  in  general, 
than  the  presence  or  absence  of  the  vertical 
object  cues.  At  this  point,  the  meaning  of  this 
effect  is  unclear,  especially  when  it  is  con¬ 
trasted  with  a  nearly  universal  pilot  preferance 
for  vertical  object  cues. 
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ABSTRACT 

A  new  generation  of  daylight  computer  image  generation  (CIG)  equipment,  based  on  tech¬ 
nology  known  as  CT-5,  employs  some  architectural  concepts  that  depart  substantially  from 
previous  practice.  A  broad  range  of  simulation  requirements  can  be  met  efficiently  by 
simple,  modular  configurations  of  equipment.  Some  interesting  characteristics  of  this 
architecture  Include  modularity  of  resolution  and  capacity,  graceful  response  to  overloads, 
channel  expansion  with  corresponding  total  capacity  growth,  and  comprehensive  treatment  of 
image  quality  Issues.  This  paper  provides  an  overview  of  CT-5  technology  and  system  character¬ 
istics  relative  to  visual  simulation  tasks. 


INTRODUCTION' 

Real-time  computer  image  generation  (CIG) 
technology,  now  well  into  its  second  decade  of 
active  development,  continues  to  be  challenged  by 
the  imagery  that  users  would  have  it  produce. 

This  is  particularly  true  in  the  case  of  daylight 
visual  systems  where  the  real  world  remains  an 
implicit  but  illusive  standard.  Nevertheless, 
effective  training  is  being  accomplished  and  these 
successes,  along  with  the  increasing  economic  im¬ 
peratives  of  training  by  simulation,  are  driving 
CIG  systems  into  more  complex  and  comprehensive 
training  application  with  corresponding  visual  re- 
qui rements . 

The  gap  between  what  is  feasible  and  what  is 
desired  can  be  successfully  bridged  only  through 
an  understanding  of  both  the  visual  system  and  the 
training  requirements  sufficient  to  merge  the  two 
into  an  effective  training  device.  This  has  often 
proven  difficult  due  in  part  to  the  fact  that 
visual  systems  have  not  been  easily  characterized 
and  their  performance  in  a  given  application  may 
be  governed  by  complex,  non-intuitive  parameters 
that  elude  meaningful  specification. 

Many  of  the  important  characteristics  of  a 
visual  system  are  a  product  of  its  architecture  - 
the  organization,  algorithms,  and  implementation 
structure  used  to  accomplish  the  image  generation 
process.  This  paper  Introduces  some  visual  system 
architectural  concepts  that  promote  the  effective 
coupling  and  straight-forward  application  of  CIG 
technology  to  a  wide  range  of  simulation  require¬ 
ments.  This  visual  system  technology,  known  as 
CT-5,  is  the  fifth  generation  in  a  series  of  con¬ 
tinuous  tone  (daylight)  systems.  The  CT-5 
architecture  represents  a  substantial  departure 
from  the  evolutionary  trend  of  the  earlier  systems 
but  is  based  on  principles  and  approaches  that 
developed  from  a  background  of  experience  on  both 
daylight  CT  and  Novoview  night/dusk  systems. 

CT-5  system  concepts  emerged  in  response  to  a 
need  for  the  simultaneous  provision  of  high  Image 
content  and  quality  across  a  large  field  of  view 
requiring  many  channels.  A  six  channel  system  has 
been  implemented  and  Is  operating  In  excess  of  its 
required  capacity  of  a  thousand  polygons  per 
channel  and  several  thousand  polygons  total. 


A  brief  system  overview  Introduces  the  main 
functional  elements  in  a  basic  system  configura¬ 
tion  and  describes  the  general  partitioning  of 
the  image  processing  functions.  We  then  focus  on 
certain  display  processing  concepts  and  examine 
their  influence  on  overall  system  characteristics. 

SYSTEM  OVERVIEW 

Figure  1  shows  a  basic  configuration  of  CT-5 
components  that  provides  a  single  channel  capa¬ 
bility  at  standard  (525  line)  resolution.  The 
major  visual  system  sections  Include  a  general 
purpose  computer,  special  purpose  hardware  called 
an  image  processor,  and  a  display. 

The  image  processor  in  this  simplest  con¬ 
figuration  consists  of  one  each  of  four  basic  unit 
types  and  would  occupy  two  standard  equipment 
racks.  The  object  manager  (OM)  and  polygon  man¬ 
ager  (PM)  units  perform  system  comnon  functions 
and  comprise  what  is  called  the  viewpoint  pro¬ 
cessor.  This  term  refers  to  the  fact  that  the 
image  processing  tasks  performed  here  relate  gen¬ 
erally  to  the  entire  scene  content  in  the  neigh¬ 
borhood  of  the  simulated  eyepoint,  rather  than  the 
particular  portions  that  pertain  to  individual 
channels.  The  channel  oriented  tasks  are  perform¬ 
ed  in  the  channel  processor  section  by  the  geo¬ 
metric  processor  (GP)  and  display  processor  (DP) 
units.  The  addition  of  system  units  to  Increase 
resolution  or  add  channels  is  accomplished  as  In¬ 
dicated  in  the  figure. 

The  system  is  structured  as  a  pipeline  of  pro¬ 
cessing  units  where  computing  tasks  are  distributed 
along  a  sequence  of  dedicated  computing  elements. 
Figure  1  also  shows  by  means  of  cross-hatched  areas 
the  location  of  several  field  buffer  memories  in 
the  system.  These  memories  are  located  at  stra¬ 
tegic  points  In  the  structure  between  major  pro¬ 
cessing  functions.  One  television  field  time  is 
allocated  to  transport  and  process  data  from  one 
buffer  to  the  next.  This  allows  the  processing 
hardware  to  operate  at  its  own  pace  and  be  designed 
to  accomnodate  only  the  average  complexity  of  an 
image  field.  An  Important  consequence  of  this 
memory  -  processor  -  memory  organization  is  that 
the  capacity  of  each  section  is  proportional  to 
the  processing  time  allotted.  The  application  of 
this  notion  to  overload  management  Is  discussed 
later. 
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Figure  1.  Basic  CT-5 

The  viewpoint  processor  contains  two  environ¬ 
ment  memories  that  provide  online  storage  for 
portions  of  the  data  base  sufficiently  close  to  the 
eyepoint  that  they  may  be  required  for  scene 
generation.  The  management  memory  Includes  struct¬ 
ural  information  about  the  data  base  that  is  used 
to  accomplish  various  data  base  management  tasks 
including  the  computation  of  what  should  be  in¬ 
cluded  in  the  environment  memories.  Details  of  the 
individual  scene  elements  such  as  point  coordin¬ 
ates.  polygon  definitions,  color,  etc.  are  contain¬ 
ed  in  the  geometric  memory. 

In  a  hierarchical  succession  of  computational 
steps,  the  viewpoint  processor  sifts  through  the 
data  base  to  extract  a  minimal  required  set  of 
scene  elements  to  produce  the  current  picture.  This 
process  includes  a  sequence  of  increasingly  dis¬ 
criminating  tests  for  field  of  view  inclusion,  and 
the  rejection  of  scene  elements  that  are  backfaclng 
or  have  a  perspective  size  less  than  a  certain 
threshold.  Thus,  the  scene  data  provided  to  a 
channel  processor  is  a  highly  refined  set  that  will 
make  efficient  use  of  the  hardware  in  this  portion 
of  the  system. 

Virtually  all  of  the  computational  power 
required  to  generate  a  television  format  image  in 
a  given  channel  is  contained  in  the  channel  pro¬ 
cessing  units  dedicated  to  that  channel.  Thus 
each  channel  tends  to  be  self-supporting  and  does 
not  interact  with  other  channels. 

The  GP  unit  performs  all  computations  involved 
in  transforming  the  incoming  model  space  descrip¬ 
tion  of  scene  elements  into  a  two-dimensional 
perspective  Image  In  display  space.  These  steps 
include  the  processes  of  rotation,  dipping,  and 
perspective  division.  The  result  of  this  process 
is  that  an  Image  plane  description  of  all  scene 
elements  that  project  on  a  display  channel  is 


System  Configuration 

stored  in  the  polygon  buffer.  The  remaining  steps 
to  generate  a  final  image  are  performed  in  the  DP 
unit  and  these  are  described  in  some  detail  in  the 
following  section. 

DISPLAY  PROCESSING  CONCEPT 

Display  processing  encompasses  the  entire  set 
of  operations  necessary  to  convert  image  plane 
descriptions  of  individual  scene  elements  Into  a 
composite  picture  with  hidden  portions  removed, 
anti-aliasing  techniques  applied,  and  TV  formatt¬ 
ing  accomplished.  These  functions  generally 
account  for  a  substantial  part  of  the  hardware  of 
real-time  CIG  systems,  and  their  implementation 
plays  a  major  role  in  determining  critical  per¬ 
formance  characteristics. 

Historically,  the  scanline  nature  of  the  dis¬ 
play  has  strongly  Influenced  the  design  of  display 
processing  hardware.  Not  only  are  the  pictures 
drawn  in  raster  format,  but  the  traditional 
architectures  tend  to  compute  images  using  scanline 
notions  as  the  conceptual  basis  for  Implementation. 
Thus,  one  generally  finds  that  images  are  computed 
in  scanline  order,  hardware  Is  partitioned  to 
accommodate  scanline  image  requirements,  and  video 
signals  are  derived  from  sample  points  obtained  by 
intersecting  scene  elements  with  mathematical  scan- 
lines.  These  notions,  although  quite  natural  in 
view  of  the  ultimate  format  of  the  video,  have 
seriously  limited  the  capacity  and  image  quality 
of  CIG  systems. 

CT-5  abandons  the  scanline  approach  to  picture 
computation.  Images  are  computed  in  feature- 
sequential  order  rather  than  in  scanline  order; 
area  processors  replace  scanline  processors; 
and  area  representations  replace  scanline  samples. 
The  basic  concepts  and  characteristics  of  this 
approach  are  described  in  the  following  sections. 
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Area  Computation 

Image  processing  Is  performed  by  hardware 
which  operates  in  parallel  on  regions  of  the  dis¬ 
play  called  spans.  A  span  is  a  rectangular  area 
In  image  space  containing  an  array  of  picture 
elements.  The  display  plane  is  divided  into  a 
contiguous  set  of  these  span  areas  that  can  be 
thought  of  as  a  coarse  pixel  set. 

Individual  scene  elements,  such  as  a  polygon, 
are  processed  a  span  at  a  time  by  sequentially 
examining  the  Involved  span  areas,  as  Indicated  in 
Figure  2.  The  span  processor  is  capable  of  handl¬ 
ing  the  most  complex  geometry  that  scene  elements 
can  generate  within  a  span.  Two  important  pro¬ 
perties  of  this  parallel  processing  approach  are 
discussed  below. 
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Figure  2.  Span  Area  Processing 

The  time  to  compute  a  span  for  a  given  scene 
element  Is  not  a  significant  function  of  the  com¬ 
plexity  of  the  scene  element  internal  to  the  span. 
Looked  at  another  way,  the  time  required  to  process 
a  scene  element  is  proportional  to  its  image  size. 

The  approach  avoids  sampling  operations.  The 
image  processing  algorithms  operate  on  analytic 
descriptions  of  the  scene  geometry  to  produce  high 
resolution  area  representations  that  preserve  the 
essential  characteristics  of  the  original  con¬ 
tinuous  image.  By  retaining  image  data  that  in¬ 
cludes  information  about  the  size,  shape,  and 
location  of  image  details  at  a  resolution  signif¬ 
icantly  higher  than  the  pixel  resolution,  is  is 
possible  to  produce  an  anti-aliased  image  of 
excellent  quality. 

Image  Formation 

Recalling  that  image  areas  are  handled  in 
span-sized  pieces,  one  can  see  how  the  principal 
processing  functions  are  accomplished  by  referlng 
to  Figure  3. 

Three  memories  participate  in  the  process. 

The  polygon  buffer  contains  Image  plane  descrip¬ 
tions  of  all  scene  elements  (such  as  polygons  and 
lights)  that  are  to  appear  In  the  final  image. 

The  scene  elements  are  arranged  In  order  of  de¬ 
creasing  visual  priority  so  that  early  submissions 
to  the  span  processor  may  occult  later  ones.  This 
dual  memory  Is  updated  at  the  field  rate  during 
normal  dynamic  operation,  but  either  side  con¬ 
tains  all  information  required  to  produce  a  com¬ 
plete  television  picture,  including  both  fields. 


Figure  3.  Display  Processing  Concept 

The  video  assembler  is  a  dual  video  field 
buffer  with  storage  for  red,  green,  and  blue  com¬ 
ponents  at  pixel  resolution.  One  side  of  this 
dual  buffer  supplies  digital  video  to  the  display 
while  the  other  half  is  used  to  assemble  the  next 
field.  The  assembly  side  of  this  memory  is  struc¬ 
tured  so  that  access  is  available  simultaneously 
to  span-sized  arrays  of  pixels,  with  provision  for 
combining  new  data  with  data  already  In  memory. 

The  mask  memory  is  used  to  store  a  high  re¬ 
solution  composite  record  of  all  prior  image 
regions  processed  in  a  field.  Only  the  geometric 
details  delineating  existence  or  absence  of  scene 
data  are  stored. 

Now  visualize  the  processing  of  a  polygon 
through  the  display  processor  structure  by  examin¬ 
ing  the  Image  sketches  accompanying  Figure  3. 
Assume  that  several  polygons,  collectively  labeled 
“A",  have  already  been  processed.  The  mask  memory 
would  contain  the  description  of  the  region 
covered  thus  far,  while  the  video  assembler  would 
contain  the  contribution  to  final  video  attribut¬ 
able  to  these  polygons  (including  visibility, 
illumination,  and  other  effects). 

Imagine  that  a  new  polygon,  "B",  is  submitted 
to  the  span  processor.  Starting  with  the  first 
touched  span,  an  analytic  description  of  the  por¬ 
tion  of  "B"  Included  in  that  span  is  formed  while 
a  corresponding  description  of  Images  already  pro¬ 
cessed  is  obtained  from  the  mask  memory.  The  data 
for  "B"  and  the  mask  memory  data  (shown  as  "A") 
are  combined  to  form  two  new  Image  descriptions: 

B  +  A  and  B  -  A.  The  image  B  A  (inclusive  OR) 
is  the  new  composite  mask  that  Is  stored  in  the 
mask  memory.  The  difference  image,  6  -  A,  is  the 
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visible  portion  of  6  that  Is  passed  to  the  spatial 
fflter  section.  Here,  the  high  resolution  geo¬ 
metric  description  Is  recombined  with  Its  associat¬ 
ed  shading  data  and  the  result  Is  spatially  filter¬ 
ed  to  derive  antl-allased  video  data  at  the  pixel 
resolution. 

The  output  of  the  spatial  filter  section  Is 
digital  video.  In  span  parallel  format,  that  re¬ 
presents  the  video  contribution  to  be  made  to  the 
scene  by  the  visible  portion  of  “B“.  The  video 
data  are  added  to  the  previous  contents  of  the 
video  assembler.  The  processing  continues  In  this 
manner  for  each  touched  span  until  the  polygon  is 
completely  processed. 

Anti-aliasing 

Several  Important  concepts  are  embodied  in  the 
anti-aliasing  techniques  used  in  CT-5.  The  Image 
presented  at  the  Input  to  the  spatial  filter  Is  a 
faithful,  high  resolution  rendition  of  the  desired 
Image,  Including  the  effects  of  visual  priority. 

All  imagery  is  treated  Identically,  without  com¬ 
plexity  limits,  and  without  directional  biases  re¬ 
lated  to  raster  orientation.  The  image  descrip¬ 
tion  has  the  desirable  property  that  a  scene 
produced  by  complex  layers  of  occulting  polygons 
will  be  identical  to  that  produced  by  an  equivalent 
input  of  polygon  tiles  forming  the  same  image,  but 
with  no  occulting  Involved. 

Spatial  filtering  is  performed  at  the  pixel 
level  as  Indicated  in  Figure  4.  The  scene  sub¬ 
mitted  to  the  filter  Includes  all  effected  areas 
of  the  screen  regardless  of  which  field  is  current¬ 
ly  being  produced.  Thus,  filtering  may  extend  to 
areas  outside  the  current  field.  Video  for  each 
pixel  is  computed  by  convolving  the  visible  scene 
portions  near  the  pixel  of  interest  with  a  two 
dimensional  filter  function.  The  filter  chosen 
is  approximately  two  pixels  wide  and  is  shaped  to 
satisfy  a  criterion  succinctly  stated  by  A.C. 

Erdahl :  “The  total  energy  contributed  to  all  dis¬ 
play  pixels  by  a  scene  fragment  should  remain 
constant  and  be  Independent  of  its  position  re¬ 
lative  to  the  pixel  structure."  This  condition 
implies  that  the  average  luminance  of  a  small 
region  of  the  display  In  the  vicinity  of  some 
particular  scene  details  is  invariant  under  motion 
of  the  scene  with  respect  to  the  raster  structure. 

A  filter  meeting  this  criterion  and  providing 
adequate  bandwidth  control  and  phase  response  has 
been  Implemented.  It  produces  Images  with  high 
subjective  resolution  and  sharpness  while  smoothly 
attenuating  the  higher  spatial  frequencies. 
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Figure  4.  Spatial  Filtering 


Capacity  Characteristics 

As  noted  earlier,  the  processing  time  for  a 
scene  element  is  proportional  to  Its  Image  area. 

It  is  Interesting  to  examine  the  relationship  be¬ 
tween  processing  time  and  incremental  image 
complexity,  given  a  basic  Image  that  we  wish  to 
embelish. 

We  can  increase  the  richness  of  a  given  scene 
in  two  principal  ways.  One  way  is  to  Increase  the 
detail  of  an  existing  object.  The  image  area  of 
the  more  detailed  object  will  be  about  the  same 
as  the  lower  complexity  version.  It  Is  not  quite 
correct  to  conclude  that  the  high  detail  object  re¬ 
quires  no  additional  processing  time.  The  In¬ 
crease  in  time  that  does  occur,  however.  Is 
primarily  related  to  the  fact  that  we  are  now  pro¬ 
cessing  smaller  polygons  which  are  less  efficient¬ 
ly  contained  by  the  span  areas.  Therefore  the 
total  number  of  spans  processed  increases  somewhat. 
However,  the  processing  time  grows  much  more 
slowly  than  does  the  polygon  count.  Thus,  one 
finds  that  increasing  polygon  counts  that  are 
attributable  to  increased  detail  in  existing  scene 
elements,  are  accompanied  by  less  than  linear  in¬ 
creases  in  computing  time. 

The  second  way  to  enhance  a  scene  is  to  add 
new  objects.  The  addition  of  objects  increases 
the  depth  complexity  of  the  picture,  i.e.  the 
average  number  of  surfaces  encountered  at  each 
point  in  the  image.  Since  we  are  forming  the  Image 
from  high  to  low  visual  priority  it  is  possible  to 
skip  over  regions  that  are  already  completely 
covered  with  scene  data.  Thus,  considerable  time 
might  be  saved  by  rapidly  detecting  "full  span" 
conditions.  Although  this  mechanism  cannot  be 
completely  efficient,  enough  help  can  be  obtained 
to  significantly  reduce  the  net  processing  cost  of 
additional  objects.  The  net  processing  time  is 
the  normal  computing  time  for  the  new  polygonal 
areas  minus  the  savings  achieved  by  more  rapid 
processing  of  areas  now  obscured.  CT-5  implements 
such  a  scheme  that  realizes  substantial  savings 
when  hidden  areas  of  scene  elements  are  encounter¬ 
ed. 

The  net  effect  of  the  principles  just  describ¬ 
ed  is  to  give  rise  to  a  capacity  vs.  processing 
time  relationship  that  has  the  general  character¬ 
istic  shown  in  Figure  5.  The  relationship  appears 
to  be  valid  over  a  reasonable  range  about  the 
nominal  system  performance.  This  characteristic 
makes  it  very  attractive  to  consider  trading  field 
time  for  image  complexity  when  needed. 


Figure  5.  OP  Capacity  Characteristic 
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Also  noteworthy  is  the  total  absence  of  mech¬ 
anisms  in  the  display  processor  architecture  that 
would  limit  scene  complexity.  It  is  possible  to 
generate  pictures  of  arbitrary  complexity  by  ex¬ 
tending  the  processing  time. 

RESULTS 

The  architectural  concepts  described  above 
lead  to  some  interesting  and  useful  system  pro¬ 
perties.  The  following  sections  summarize  the 
resulting  system  characteristics  that  may  be  of 
particular  interest  to  those  who  plan,  specify  and 
apply  CIG  systems. 

Overload  Response 

Overload  in  CIG  visual  systems  is  inevitable. 
Regardless  of  the  specified  capacity  of  a  system, 
the  user  will,  in  time,  exceed  it. 

The  manner  in  which  a  system  responds  to  over¬ 
load  is  very  important  to  the  user.  Two  systems 
with  the  same  nominal  capacity  can  prove  to  be 
quite  different  in  practice  when  compared  on  the 
basis  of  effective  performance.  Some  systems  are 
characterized  by  having  absolute  capacity  limits 
which  are  at  or  near  their  nominal  performance 
limits.  Breaching  these  limits,  even  by  a  small 
amount,  can  result  in  drastic  image  changes.  The 
only  available  strategy  in  such  cases  is  to  tune 
the  model  so  that  overload  does  not  occur  --  even 
briefly.  This  tuning  is  accomplished  typically  by 
limiting  model  density  and  by  carefully  structur¬ 
ing  the  level  of  detail  transitions  to  prevent 
overload  -  a  costly  undertaking  that  generally  re¬ 
sults  in  under-utilization  of  the  system. 

CT-5  systems  automatically  manipulate  image 
generator  resources  to  produce  controlled  re¬ 
sponses  over  a  broad  range  of  impending  and  actual 
overload  conditions.  Loads  on  the  order  of  twice 
the  nominal  image  capacity  can  be  accommodated 
frequently  without  catastrophic  losses  in  the 
imagery.  Thus,  the  system  may  be  operated  at  much 
higher  average  load  levels,  and  model  tuning  re¬ 
quirements  can  be  relaxed  or  adjusted  to  suit  the 
training  mission  needs.  This  fortunate  impact  on 
modeling  requirements  will  become  increasingly 
important  with  the  advent  of  larger  and  larger 
data  bases  as  automated  modeling  techniques  are  em¬ 
ployed  and  hand  tuning  becomes  prohibitive. 

Three  principal  mechanisms  are  employed  to 
accomplish  load  management  objectives: 

1.  Cl-5  capacity  Increases  with  available 
computing  time.  This  characteristic, 
previously  described  for  the  display  pro¬ 
cessing  functions,  also  holds  for  the 
remainder  of  the  system,  where  the  only 
limits  beside  computing  time  are  certain 
memories,  and  they  are  generously  sized 
relative  to  nominal  requirements. 

2.  Polygons  are  rejected  based  on  their  per¬ 
spective  size.  This  size  threshold  can 
be  adjusted  to  provide  continuous  control 
over  the  number  of  polygons  presented  to 
the  pipeline. 


3.  Overall  scene  loading  can  be  controlled 
by  level -of-detail  selections  and  the 
range  thresholds  at  which  switching 
occurs.  These  thresholds  can  be  changed 
dynamically. 

The  general  purpose  computer  monitors  load  at 
various  points  in  the  image  processor  and  also 
controls  the  field  time.  Under  normal  conditions 
the  field  time  is  kept  constant.  Impending  over¬ 
load  conditions  are  sensed  and  regulated  by  chang¬ 
ing  the  thresholds  on  perspective  size  and  level 
of  detail. 

If  scene  conditions  should  change  so  rapidly 
that  units  cannot  complete  their  tasks  in  the 
allotted  field  time,  then  the  field  time  is  simply 
extended  to  provide  immediate  relief  for  the 
problem.  Thus  the  instantaneous  penalty  for  over¬ 
load  is  a  lower  field  rate. 

Field  rate  reduction  is  viewed  as  a  temporary 
measure  to  be  used  while  more  gradual  adjustments 
are  made  on  level  of  detail  and  perspective  size 
parameters.  These  changes  reduce  the  load  by 
rejecting  increasingly  larger  polygons.  Thus  the 
long  term  penalty  is  that  potentially  perceptible 
polygons  are  rejected  and  alternate  scene  defini¬ 
tions  come  into  use  sooner. 

At  intermediate  values  of  these  thresholds  it 
becomes  preferable  to  accept  operation  at  a  lower 
field  rate  rather  than  to  reject  scene  details  of 
obvious  and  useful  size.  The  principal  effect  of 
a  lower  field  rate  is  to  Introduce  the  possibilty 
of  perceptible  flicker.  Field  time  extension  is 
therefore  limited  to  preclude  excessive  flicker. 

Under  still  more  severe  conditions  the 
strategy  resorts  to  frame  rate  update.  This 
doubles  the  processing  time  and  capacity  for  all 
units  up  to  the  display  processor.  As  a  final 
measure,  the  display  processor  can  resort  to  a 
frame  rate  update  cycle  with  field  rate  display. 

In  suninary,  the  overload  response  of  CT-5  is 
designed  to  fit  the  penalty  to  the  crime.  The 
result:  graceful  overload  response  and  more  use¬ 
ful  Imagery  on  the  screen  more  of  the  time. 

Image  Quality 

One  of  the  principal  objectives  for  CT-5  was 
to  improve  image  quality  well  beyond  that  avail¬ 
able  from  the  best  existing  CIG  systems.  Histori¬ 
cally,  the  CT  system  series  has  pioneered  in  this 
area  and  substantial  progress  has  been  achieved. 

(1)  Our  continuing  commitment  to  image  quality 
improvements  is  based  on  the  following  convictions: 

1.  Tangible  benefits  in  system  effectiveness 
are  associated  with  the  production  of 
high  quality  Images  with  a  minimum  of  dis¬ 
tracting  visual  noise. 

2.  We  are  not  in  any  Immediate  danger  of 
having  solutions  that  exceed  the  needs. 

The  CT-5  system  approach  implements  a  funda¬ 
mentally  sound  and  comprehensive  solution  to 
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aliasing  problems  by  the  techniques  outlined 
earlier.  In  the  course  of  developing  and  evaluat¬ 
ing  these  concepts  we  designed  a  test  pattern  that 
was  found  to  be  quite  descriminating  in  issues  of 
image  quality.  A  photograph  of  the  pattern  as 
generated  by  a  CT-5  system  on  a  700  line  color 
display  is  shown  in  Figure  6. 

The  pattern  consists  of  triangles  radiating 
from  a  common  center.  The  triangles  are  defined 
as  maximum  intensity  polygons  against  a  black  back¬ 
ground.  At  the  periphery  of  the  pattern  each 
triangle  is  one  pixel  wide  and  the  space  between 
neighboring  triangles  is  four  pixels.  The  pattern 
reveals  much  about  system  performance. 

The  first  thing  to  note  from  the  photograph  is 
that  there  is  very  little  variation  in  the  appear¬ 
ance  of  the  triangular  spokes  as  a  function  of 
their  orientation.  The  system  tends  to  treat  scene 
details  equally,  without  directional  biases. 

The  individual  spokes  are  resolvable  towards 
the  center  well  beyond  the  half-way  point.  Thus 
we  see  that  the  system  is  able  to  consistently  re¬ 
solve  detail  that  measures  less  than  one  half  pixel 
in  size  and  the  detail  appears  in  proper  positional 
relationship  (phase)  with  similar  neighboring 
elements.  This  aspect  of  the  pattern  indicates 
that  high  effective  resolution  is  achieved  and 
that  small,  generally  isolated  segments  of  scene 
detail  can  be  visually  tracked  down  to  fractional 
pixel  size. 

The  final  observation  about  the  test  pattern 
performance  is  that  the  spokes  blend  uniformly 
into  the  central  region.  It  is  worth  noting  that 
the  fundamental  spatial  frequency  of  the  pattern 
is  at  the  theoretical  Nyquist  limit  of  one  raster 
line  pair  at  a  point  40%  out  from  the  center  of 
the  pattern.  This  is  the  approximate  point  at 
which  the  pattern  blends  together.  The  central 
part  of  the  pattern  maintains  a  generally  constant 
character  and  average  intensity.  Pattern  modula¬ 
tion  decreases  uniformly  and  no  coherent  patterns 
arise  toward  the  central  area  as  would  be  charac¬ 
teristic  of  aliased  spectra.  This  propery  allows 
small  objects  to  be  portrayed  and  recede  unobtru¬ 
sively  into  their  background. 

One  important  issue  not  addressed  by  the 
pattern  is  that  form  of  temporal  aliasing  encoun¬ 
tered  in  an  interlaced  display  system.  The 
tendency  of  the  eye  to  track  scene  motion  and  cause 
the  field  lines  to  appear  to  lie  on  top  of  one 
another  aggravates  aliasing  behavior.  Although 
field  tracking  is  a  fundamental  problem  in  inter¬ 
laced  displays,  the  attendent  aliasing  effects  can 
be  reduced  by  broadening  the  spatial  filter  func¬ 
tion  to  include  contributions  from  areas  outside 
the  current  field.  The  CT-5  system  uses  image  data 
from  all  regions  of  the  screen  to  form  an 
individual  field. 

The  use  of  a  video  buffer  that  stores  image 
data  individually  for  each  pixel  has  made  it 
feasible  to  Implement  other  subtle  improvements  in 
image  quality.  Previously,  scanline  approaches  to 
picture  generation  have  encoded  video  data  as 
raster  line  segments  that  define  only  linear 
changes  in  color  components.  As  a  result,  anti¬ 
aliasing  and  shading  changes  due  to  limited  visi¬ 
bility  had  to  be  approximated  by  linear  changes  in 


video  signals.  This  approximation  is  Incorrect 
and  troublesome  for  low  visibility  effects  where 
it  leads  to  some  serious  visual  anomalies.  CT-5, 
on  the  other  hand,  is  free  from  these  restrictions 
since  results  are  computed  independently  for  each 
pixel.  The  pixel  buffer  removes  all  constraints 
on  the  ultimate  complexity  of  the  stored  image. 

Modularity 

The  CT-5  architecture  provides  considerable 
flexibility  to  expand  the  basic  system  config¬ 
uration  to  meet  various  requirements  for  number  of 
channels  and  for  TV  resolution.  Interaction 
among  these  expansion  options  is  negligible, 
allowing  one  to  configure  systems  without  having 
to  juggle  complex  performance  tradeoffs.  The 
hardware  additions  consist  of  units  identical  to 
those  used  in  the  baseline  configuration. 

Channel  Expansion.  Channel  expansion  in  CT-5 
is  achieved  by  the  addition  of  channel -specific 
parallel  hardware  at  an  early  stage  in  the 
processing  pipeline.  These  modular  system 
increments  provide  independent  computing  capacity 
for  each  channel,  thus  decoupling  the  performance 
of  any  one  channel  from  that  of  the  other  channels. 

The  parallel  channel  approach  used  in  CT-5 
contrasts  with  the  organization  commonly  employed 
by  most  other  C1G  systems,  wherein  a  single  image 
processing  resource  is  shared  among  all  channels. 
That  architecture  minimizes  the  channel  specific 
hardware,  but  the  available  system  capacity  must 
be  divided  in  direct  proportion  to  the  number  of 
channels  served.  Thus,  both  model  and  scene 
densities  must  be  reduced  as  channels  are  added. 
Furthermore,  secondary  system  constraints  such  as 
edge  crossings  per  scanline  or  total  segments  often 
remain  constant  with  increasing  channels  and  become 
serious  limits  when  applied  to  the  composite  of  all 
channels. 

The  CT-5  approach  was  motivated  by  a  desire  to 
serve  efficiently  both  single  channel  requirements 
as  well  as  wide  field  of  view,  multi-channel  con¬ 
figurations,  without  having  to  sacrifice  scene 
density.  In  operational  terms  this  means  that 
a  model  designed  for  a  single  channel  CT-5  system 
will  also  be  of  about  the  right  complexity  for  a 
multi-channel  system. 

Having  independent  parallel  channels  has  other 
advantages.  For  example,  the  conduct  of  simulta- 
eous  independent  simulations  with  one  CT-5  visual 
system  is  possible  without  the  danger  of  unpredict¬ 
able  interactions  among  the  simulations.  CT-5  is 
designed  to  accommodate  a  variety  of  other  special 
requirements  of  this  type.  Independent  channels 
also  greatly  simplify  fault  isolation  and  make  it 
highly  probable  that  failure  effects  will  be  con¬ 
fined  to  a  single  channel  and  allow  training  to 
continue  on  the  remaining  channels. 

Resolution.  Increased  raster  resolution  is 
accomplished  by  the  modular  addition  of  display 
processing  units.  The  single  DP  unit  used  in  the 
basic  configuration  provides  a  raster  of  1/4 
million  pixels  -  equivalent  to  a  525  line  T  V 
standard.  Expansion  to  700  line  resolution  has 
been  accomplished  and  provision  has  been  made  to 
accommodate  1000  line  requirements.  The  relative 
distribution  of  available  pixels  between  lines 


Figure  6.  Aliasing  Test  Pattern  Generated  on  CT-5 


Figure  7,  A  180  Polygon  Aircraft  Scene 
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and  elements  is  quite  flexible,  thus  permitting  a 
wide  variation  in  the  aspect  ratio  of  the  chosen 
display.  OP  units  are  designed  to  operate  in 
parallel  configurations  that  divide  the  span  pro¬ 
cessing  load  so  that  there  is  no  attendent  sacri¬ 
fice  in  channel  capacity  as  resolution  is  in¬ 
creased. 

In  addition  to  the  above  advantages,  the  re¬ 
lative  Independence  of  channel  processing  hard¬ 
ware  even  permits  different  scan  standards  to  be 
used  in  different  channels.  This  capability  would 
be  Important  in  the  future  if  the  application  of 
wide  angle  optics  in  selected  channels  prompts  the 
need  for  high  pixel  counts  in  some  channels  and 
standard  counts  in  others  while  preserving  con¬ 
sistent  scene  complexity  and  image  quality  across 
all  channels. 

CONCLUSION 

A  new  CIG  system  architecture  has  been  de¬ 
scribed.  It  includes  some  novel  concepts  that 
offer  effective  solutions  to  perenlal  problems  en¬ 
countered  in  real-time  visual  systems. 

CT-5  applies  these  concepts  in  a  set  of  system 
modules  that  implement  a  modular,  efficient,  and 
extensible  approach  to  the  generation  of  scenes 
composed  of  polygons  and  lights.  These  same 
principles  also  extend  quite  naturally  to  more 
complex  scene  primitives  and  to  texture. 

Parallel  area  processing  is  the  central  theme 
In  the  conceptually  straight-forward  implementa¬ 
tion  of  display  processing  functions  found  in  CT-5. 


The  concepts  related  to  area  processing  are  an 
integral  part  of  the  image  quality,  capacity,  and 
growth  characteristics  of  the  system. 

The  architectural  properties  that  equate  pro¬ 
cessing  time,  degree  of  area  parallelism,  and 
capacity  are  perhaps  most  intriquing.  For  the 
present,  this  has  made  possible  the  provision  of 
an  extremely  elastic  overload  response  within  the 
context  of  a  high  basic  capacity.  In  the  future, 
systems  will  undoubtedly  tap  increases  in  either 
computing  speed,  parallelism,  or  both  to  achieve 
their  goals;  and  the  trend  toward  simple, 
repetitive  architectural  structures  will  continue 
with  the  further  application  of  LSI  technology. 
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ABSTRACT 

A  NEW  APPROACH  TO  CGI  SYSTEMS:  A  new  approach  to  CGI  visual  systems  is  presented  which 
has  distinct  advantages  over  conventional  systems.  Although  conventional  systems  have  pro¬ 
vided  the  simulation  community  with  a  high  level  of  sophistication,  they  are  costly  and 
lack  the  modularity  required  for  low-cost,  limited  requirement  trainers. 

The  new  CGI  approach  capitalizes  on  recent  advances  made  in  modern  semiconductor  tech¬ 
nology  coupled  with  double-buffered  refresh  memory,  video  lookup  tables,  and  modular  design 
techniques.  The  advantages  of  this  new  system  (GVS-1)  are  reduced  system  cost,  minimal 
development  risk,  high  modularity,  and  high  reliability. 

The  basic  components  of  this  system  (general  purpose  computer,  geometric  processor, 
display  generator,  illumination  control)  are  described  in  this  paper.  In  addition, 
texturing  capability  of  the  system  is  briefly  discussed. 


CONVENTIONAL  CGI  VISUAL  SYSTEMS 

The  past  decade  has  been  one  of  rapid  growth 
in  CGI  visual  systems.  During  this  time,  two 
major  technological  solutions  evolved  in  the  form 
of  low-cost  calligraphic  and  high-cost  raster  scan 
approaches.  The  former,  developed  originally  to 
present  simple  night  landing  and  takeoff  scenes, 
have  been  expanded  recently  to  offer  daytime 
scenarios.  They  are,  however,  still  basically 
low  capability  systems  limited  in  scene  detail  and 
number  of  colors.  The  latter,  created  to  meet 
much  more  demanding  training  requirements  (high 
detail  with  full  color),  have  achieved  a  signifi¬ 
cant  level  of  sophistication.  These  systems  can 
produce  realistic  pictures  of  man-made  objects 
(which  can  easily  be  represented  by  edges)  and 
somewhat  less  realistic  images  of  natural  terrain 
and  vegetation.  Unfortunately,  the  price  of  con¬ 
ventional  raster  scan  systems  is  such  that  only  a 
few  trainer  applications  can  justify  their  use. 
This  high  cost,  which  is  a  direct  result  of  their 
complexity,  can  be  attributed  to  the  level  of 
technology  available  at  the  time  of  their  initial 
development. 

The  basic  concept  and  design  stage  of  conven¬ 
tional  raster  scan  systems  occurred  a  number  of 
years  ago,  when  the  lack  of  fast  and  inexpensive 
mass  storage  devices  precluded  any  method  incor¬ 
porating  intermediate  digital  storage  of  the 
entire  screen  image.  This,  in  turn,  forced  these 
systems  into  defining  the  image  "on  the  fly"  in 
synchronization  with  the  raster  TV  display,  and 
placed  an  enormous  computational  burden  upon  the 
system  design.  Not  only  was  there  the  obvious 
requirement  of  defining  which  objects  should 
appear  on  the  screen,  but  also  in  which  order 
these  objects  should  appear  in  terms  of  the  left- 
to-right,  top-to-bottom  raster  format.  This 
ordering,  or  sorting,  was  the  primary  factor  lead¬ 
ing  to  the  high  complexity  and  cost. 


THE  NEW  APPROACH 


The  recent  advances  made  in  semiconductor 
technology  have  resulted  in  the  opportunity  to 
design  and  construct  a  CGI  raster  visual  system 
in  a  completely  new  manner.  The  availability  of 
cheap,  fast,  and  dense  mass  storage  devices, 
combined  with  the  derivation  of  new  algorithms 
implemented  in  hardware  and  firmware,  has  allowed 
this  technology  to  be  applied  in  an  extremely 
efficient  manner. 

All  raster  CGI  systems  perform  scan  conver¬ 
sion,  the  process  of  converting  a  picture  image 
into  lines  and  picture  elements  for  presentation 
on  a  TV  screen.  Unlike  conventional  systems 
which  perform  this  function  by  multiple  sorting 
operations,  the  use  of  double-buffered  image 
memories  allows  the  scan  conversion  to  be  accom¬ 
plished  by  merely  reading  the  image  out  in  a 
raster  order  after  it  had  previously  been  stored 
in  a  convenient  random  fashion. 

The  Gould  Visual  System,  GVS-1,  developed  at 
Gould  Simulation  Systems  Division,  is  the  result 
of  this  progress  and  embodies  both  the  simplicity 
and  low  cost  of  calligraphic  systems  and  the  high 
performance  of  conventional  raster  scan  systems. 
Among  the  advantages  of  the  GVS-1  are: 

•  Simplified  architecture  and  standard  parts 

•  Less  hardware  and  high  modularity 

•  Low  cost 

•  High  reliability 

The  implementation  of  mass  semiconductor  stor¬ 
age  organized  into  memory  planes  plus  unique 
algorithms  to  make  full  use  of  the  memory  plane 
approach  provides  for  a  simple  and  straightforward 
system.  Another  advantage  is  the  repeatability  of 
blocks  where,  for  example,  all  memory  boards  are 
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identical.  Thus,  a  major  advantage  of  the  GVS-1 
is  in  its  simplified  architecture  and  standard 
parts. 

Less  total  hardware  exists  in  the  GVS-1  than 
conventional  systems,  since  the  use  of  the  mass 
storage  technique,  combined  with  powerful 
algorithms,  has  eliminated  the  amount  of  compo¬ 
nents  required  to  perform  the  necessary  CGI 
functions.  High  modularity,  a  primary  design 
criteria,  was  achieved  as  a  direct  result  of  the 
simplicity  of  the  system,  and  provides  ease  of 
interchangeability,  fault  isolation,  and 
maintenance. 

Low  cost  is  inherent  because  of  the  system's 
simplicity  and  low  parts  count.  Low  initial 
costs  and  low  life  cycle  costs  are  a  result  of 
using  semiconductor  technology  which  is  continu¬ 
ally  going  down  in  cost.  Therefore,  future 
replacement  of  parts  will  be  less  costly  with 
time.  Other  systems  not  incorporating  this 
approach  do  not  have  the  same  cost  advantage. 

Another  advantage  resulting  from  this  approach 
is  high  reliability.  The  GVS-1  uses  standard 
grade  circuit  elements  and  operates  well  within, 
the  operating  range  of  each  of  these  elements. 


GVS-1  SYSTEM  OVERVIEW 

The  GVS-1  visual  system  consists  of  the  follow¬ 
ing  major  subsystems  as  illustrated  in  Figure  1. 


•  General  purpose  computer 

•  Geometric  processor 

•  Display  generator 

•  Illumination  control 

•  CRT  monitor 


The  general  purpose  computer  provides  the 
interface  between  the  visual  system  and  the  rest 
of  the  simulator.  It  performs  scene  data  base 
access/retrieval  functions  and  supplies  data  and 
control  parameters  to  the  other  GVS-1  subsystems. 
These  special  purpose  hardware  units  calculate 
and  produce  the  simulated  scene  for  display  on 
the  CRT  monitor.  The  geometric  processor  performs 
three-dimensional  coordinate  transformations  and 
defines  the  instantaneous  scene  image  in  a  form 
ready  for  use  by  the  display  generator.  The  dis¬ 
play  generator  converts  the  digitally  defined 
scene  into  RGB  video.  During  this  operation, 
simultaneous  write  and  read  operations  are  per¬ 
formed  within  the  double  buffer  image  memories. 
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Figure  1.  GVS-1  Functional  Allocation  And  Timing 
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As  the  data  list  from  the  geometric  processor 
is  being  stored  into  one  set  of  memory  planes, 
the  other  buffer  (just  previously  completed)  is 
read  and  the  output  scene  image  resolved  and 
converted  to  analog  video.  The  final  image  pro¬ 
cessing  is  performed  by  the  illumination  control 
which  enhances  the  scene  realism  by  providing 
such  effects  as  horizon  glow,  fading  of  colors 
with  distance,  etc.  Finally  the  completed  image 
is  transferred  to  the  CRT  monitor  for  display. 

The  system  performance  parameters  are  pre¬ 
sented  in  Table  1.  The  modular  nature  of  the 
design  provides  great  flexibility  and  modifica¬ 
tion  of  many  parameters  can  be  accomplished  with 
only  simple  hardware  changes. 

GENERAL  PURPOSE  COMPUTER 

The  general  purpose  computer  provides 
the  data  and  control  information  for  the 
rest  of  the  visual  system  as  a  result  of  com¬ 
munication  with  the  simulator.  It  primarily 
performs  bookkeeping  functions  and  is  not 
involved  with  the  real  time  image  geometri¬ 
cal  transformantions  which  are  executed  in 
separate  special  purpose  hardware 
(geometric  processor). 

GEOMETRIC  PROCESSOR 

The  geometric  processor  is  the  major  compu¬ 
tational  element  of  the  GVS-1  visual  system.  It 


utilizes  inputs  from  the  general  purpose  computer 
and  performs  all  image  transformation  calculations 
required  to  convert  three-dimensional  scene 
elements  into  a  form  compatible  with  the  display 
generator. 

General  Concept 

The  geometric  processor  is  a  high-speed, 
parallel  processing  device  based  upon  large  scale 
integrated  (LSI)  circuit  elements.  Groups  of 
these  elements  are  organized  into  a  distributed 
processing  network  to  accomplish  the  necessary 
arithmetic  and  logical  functions.  The  configura¬ 
tion  utilizes  two  basic  computational  structures  - 
scalar  processors  (SPs)  and  vector  processors 
(VPs).  The  scalar  processors  interface  external 
devices,  perform  general  purpose  functions  and 
provide  data  and  control  for  the  vector  processors. 
The  vector  processors  perform  high-speed  repeti¬ 
tive  arithmetical  and  logical  operations.  Over¬ 
lapped  execution  ("pipelining")  is  used  to 
achieve  high  throughput  rates  and  to  allow  the 
geometric  processor  operations  to  occur  simul¬ 
taneously  with  the  display  generator  image  draw¬ 
ing.  As  a  result,  image  generation  in  the  GVS-1 
is  a  two-frame  operation.  The  first  frame  con¬ 
sists  of  calculation  and  storage  and  includes  both 
the  geometric  processor  functions  and  the  display 
generator  drawing.  The  second  frame  is  a  reading 
frame,  during  which  the  image  is  displayed  on 
the  CRT  monitor.  The  ability  to  fully  develop 
and  display  an  image  within  only  two  TV  frames  re¬ 
presents  an  important  feature  of  the  system. 


TABLE  1.  GVS-1  SYSTEM  PERFORMANCE 


BASIC  SYSTEM 

— 

OPERATING  OPTIONS 

SYSTEM  OPTIONS 

2000  potentially  visible  edges 
(32  pixel  average  edge  length) 

1600  visible  edges  with  1000 
light  points 

20K  edge  data  base 

Dynamic  occulting  priorities 

64  instantaneous  colors  from 
a  color  base  of  32K  colors 

Dawn,  day,  dusk,  night  scene 
illumination  levels 

Edge  smoothing 

Weather  -  range  visibility 
effects 

Moving  models 

Full  perspective  presentation 
moving  observer 

Four  levels  of  detail 

512  visible  raster  lines 

30  Hz,  2:1  interlace 

Update  rate 

30,  15,  10,  5  Hz 

640  visible  raster 
lines 

Number  of  channels 

Curved  surface 
shading 

Larger  data  base 

Texture 

Functional  Description 

The  general  purpose  computer  provides  simula¬ 
ted  scene  data  (terrain  and  flying  models)  in  the 
form  of  edge  endpoints  to  the  geometric  processor 
These  data  are  expressed  in  their  own  coordinate 
systems  with  known  relationships  to  the 
observer's  instantaneous  coordinate  system.  The 
geometric  processor  transforms  these  data  to 
obtain  the  instantaneous  simulated  scene  in 
screen  coordinates.  The  geometric  processor  pro¬ 
vides  hardware  implemented  high-speed  arithmetic 
and  logic  operations  necessary  for  the  real  time 
execution  of  the  following  algorithms: 

•  Oynamic  occultation  priority  list 
generation 

•  Face  visibility  determination 

•  3-0  translation  and  rotation 

•  3-D  clipping  and  true  projection 

•  Pseudo-edge  generation 

Figure  2  depicts  the  distribution  of  these 
algorithms  within  the  computation  structure  of 
the  geometric  processor. 

The  dynamic  occultation  priority  list 
algorithm  is  performed  for  all  three-dimensional 


convex  solid  objects  (stationary  and  moving) 
which  may  occult  one  another.  This  process  is 
implemented  for  any  portion  of  the  simulated 
scene  which  was  chosen  for  viewing  at  a  given 
frame  time. 

Dynamic  occultation  priority  lists  for 
stationary  objects  (mountains,  cultural  features, 
etc.)  and  for  moving  models'  components  (aircraft 
body,  wings,  elevators,  etc.)  as  viewed  by  the 
observer  are  created  separately.  Then  the 
occulting  powers  of  the  moving  models  relative  to 
the  terrain  objects  are  determined.  This  allows 
the  merging  of  the  separate  priority  lists  into 
the  total  dynamic  occultation  priority  list  for 
all  stationary  and  moving  objects. 

Face  visibility  determination  is  performed  to 
eliminate  those  faces  of  the  three-dimensional 
convex  solid  objects  which  are  invisible  to  the 
observer  at  a  given  instant  of  time.  All  other 
faces  which  are  visible  can,  however,  be  occulted 
by  objects  which  are  closer  to  the  observer.  The 
ordering  of  these  occulting  objects  is  determined 
by  a  priority  list  of  dynamic  occultation.  Face 
visibility  is  established  by  examining  the  angle 
between  the  outward  normal  vector  of  the  face 
and  the  vector  from  the  eyepoint  to  any  point  on 
this  face  where  both  vectors  are  expressed  in  the 
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Figure  2.  GVS-1  Geometric  Processor  Algorithm  Distribution 
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observer's  coordinate  system.  The  face  is 
visible  if  the  angle  between  these  two  vectors 
is  less  than  90  degrees. 

Three-dimensional  translation  and  rotation  is 
performed  to  transform  the  terrain  and  moving 
models  edge  points  from  their  own  coordinate 
systems,  defined  in  the  data  base,  to  the 
observer’s  instantaneous  coordinate  system.  This 
algorithm  accomplishes  the  translation  and  rota¬ 
tion  for  the  general  case  of  an  observer  and  a 
model  or  models  moving  independently  with  respect 
to  a  fixed  Earth  coordinate  system. 

Clipping  is  performed  to  eliminate  those 
edges  and  parts  of  edges  describing  the  simulated 
scene  that  are  outside  the  observer's  instan¬ 
taneous  field  of  view.  Subsequent  to  clipping, 
true  perspective  projection  of  the  image  into 
the  screen  coordinate  system  is  performed. 

After  execution  of  the  previous  algorithms  of 
face  visibility  determination  and  three- 
dimensional  rotation  and  translation,  the  simu¬ 
lated  scene  is  defined  in  terms  of  connecting 
points  {edge  endpoints)  whose  locations  are 
described  in  the  observer's  coordinate  system. 

All  invisible  faces  have  already  been  excluded 
from  the  scene  and  a  priority  list  of  dynamic 
occultation  was  set  up  for  future  execution  in 
the  display  generator. 

Assuming  that  the  viewing  window  (screen) 
is  a  rectangle,  perpendicular  to  the  X  axis, 
and  located  at  a  known  distance  from  the 
origin  of  the  observer's  coordinate  system,  a 
pyramid  of  visibility  is  defined  with  its 
vertex  at  this  origin.  All  edges  and  portions 
of  edges  describing  the  simulated  scene  out¬ 
side  the  pyramid  walls  on  the  positive  side 
of  the  pyramid  are  rejected  as  invisible  be¬ 
fore  calculating  the  true  perspective  division. 

In  order  to  display  full  color  faces  in 
realtime,  each  face  of  the  simulated  scene  is 
automatically  painted  (filled)  by  the  display 
generator  using  left  side  edges  to  turn  the 
raster  on  and  right  edges  to  turn  it  off. 
Pseudo-edges  are  generated  along  the  left  side 
of  the  viewing  window  for  those  visible  faces 
which  cross  the  left  boundary  of  the  screen. 

Since  the  clipping/true  projection  algorithm 
removes  portions  of  these  faces  as  being  out¬ 
side  the  viewing  pyramid,  these  artificial 
edges  replace  the  clipped-out  edges  and  enable 
the  raster  scan  system  to  paint  the  remaining 
visible  portion  of  the  face  correctly. 


01  SPLAY  GENERATOR 


The  display  generator  is  the  key  element  of 
the  GVS-1  visual  system.  It  utilizes  digital 
inputs  from  the  general  purpose  computer  and 


geometric  processor  and  provides  outputs  of  full 
color  analog  video.  This  video  is  the  input  to 
the  illumination  control  where  visibility  effects 
are  added  prior  to  display  on  the  CRT  monitor. 

General  Concept 

The  display  generator  is  based  upon  the  con¬ 
cept  of  using  high-speed  random  access  semicon¬ 
ductor  mass  storage  devices  organized  in  a  double¬ 
buffer  manner.  Each  buffer  consists  of  multiple 
"memory  planes"  composed  of  these  devices.  Each 
memory  plane  represents  a  bit  map  of  the  entire 
display  screen.  The  buffers  are  used  simultane¬ 
ously  to  store  (write  cycle)  or  to  retrieve 
(read  cycle)  an  entire  frame  of  image  information, 
reversing  their  roles  on  a  frame-by-frame  basis. 
After  the  retrieval  process  is  completed  for  a 
given  buffer,  its  memory  planes  are  clear  (all 
data  bits  set  to  zero)  and  are  ready  to  accept 
data  for  the  next  frame.  At  the  same  instant,  the 
other  buffer  has  completed  Its  storage  process  and 
is  ready  to  start  its  read  cycle  for  the  present 
frame.  This  implementation  thus  allows  new  data 
to  be  written  into  one  buffer  while  the  other  is 
being  read  out  in  synchronism  with  the  TV  monitor 
raster.  A  functional  block  diagram  of  the  dis¬ 
play  generator  is  presented  in  Figure  3. 


The  semiconductor  memory  planes  can  be 
rapidly  written  into  and  read  out  in  a  completely 
random  manner.  The  GVS-1  visual  system  however, 
utilizes  this  feature  in  a  specific  way.  The 
storage  or  writing  operation  is  performed  in  a 
convenient  random  fashion,  while  the  read  opera¬ 
tion  is  ordered  into  the  required  TV  raster 
format.  The  double-buffer  mechanism  and  sepa¬ 
rate  read  and  write  logic  allows  for  parallel 
execution  of  these  two  operations,  resulting  in 
both  low  transport  delay  time  and  high  scene 
detail  for  a  given  update  rate. 


The  ability  to  randomly  store  the  image  data 
and  to  subsequently  retrieve  it  automatically  in 
a  raster  order  constitutes  a  significant  advan¬ 
tage  of  the  GVS-1  over  conventional  visual 
systems  of  similar  performance.  It  eliminates 
the  necessity  of  multiple  sorting  operations  and 
multiple  vector  generators,  and  greatly  simplifies 
overload  management  operations  by  eliminating 
such  constraints  as  the  limit  on  the  number  of 
edge  crossings  per  each  raster  scan  line,  etc. 

In  addition,  the  separation  of  the  scene  develop¬ 
ment  process  into  two  distinct  write  and  read 
cycles  provides  new  ''breathing  space"  in  critical 
image  enhancement  operations.  For  example,  in 
conventional  raster  visual  systems,  edge  smooth¬ 
ing  must  be  completely  calculated  and  executed 
within  the  pixel  display  time.  In  GVS-1,  edge 
smoothing  is  divided  into  two  stages;  smoothing 
information  is  developed  and  stored  during  the 
write  cycle  and  executed  during  the  read  cycle, 
thus  relaxing  the  constraints  on  the  time  devoted 
to  this  operation. 
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Figure  3.  GVS-1  Display  Generator  Functional  Block  Diagram 


Functional  Description 

The  display  generation  functions  fall  into 
two  major  categories  -  write  cycle  operations  and 
read  cycle  operations.  The  two  cycles  occur 
simultaneously  in  separate  partitions  of  double- 
buffered  refresh  memory. 

During  the  write  cycle  a  correct  geometrical 
description  of  an  instantaneous  picture 
(corresponding  with  a  given  frame)  is  stored  in 
the  selected  buffer.  The  manner  in  which  the 
information  is  written  into  the  individual 
memory  planes  of  the  buffer  provides  the  subse¬ 
quent  read  cycle  operations  with  the  necessary 
data  to  display  the  picture  in  full  color  with 
proper  occultation  and  edge  smoothing. 

The  write  cycle  operations  include: 

•  Face  edge  outline  generation 

•  Antistreaking  resolution 

•  Face  edge  priority  storage 

•  Face  edge  smoothing  factor  generation 

Face  edge  outline  generation  is  performed 
using  the  scene  data  list  received  from  the  geo¬ 
metric  processor.  This  list  contains  face  edge 
endpoints  expressed  in  the  screen  coordinate 
system  with  a  subpixel  precision  along  each 
axis.  The  geometric  processor  is  therefore 
able  to  position  a  face  with  higher 


resolution  than  that  of  the  CRT  monitor.  The 
vector  generator,  working  in  subpixel  increments 
utilizes  this  precision  when  drawing  the  face  edge 
outline  to  calculate  and  store  smoothing  factors. 
Combined  vertical  and  horizontal  edge  smoothing 
is  then  executed  during  the  read  cycle  to  reduce 
the  digital  aliasing  effects  of  apparent  angular 
rotation,  edge  crawling,  and  small  surface 
scintillation. 

The  vector  generator  extends  a  line  one  full 
pixel  at  a  time  between  each  pair  of  face  edge 
endpoints.  At  each  pixel  boundary  a  comparison 
is  performed  between  the  previously  stored  data 
and  the  new  edge  information.  The  result  of  this 
comparison  is  the  removal  of  those  edges  or  por¬ 
tions  of  edges  which  would  cause  erroneous 
streaks  of  color  in  the  final  displayed  image. 

Face  edge  data  that  remains  allows  a  simple  read 
cycle  mechanism  to  fill  faces  with  colors.  The 
particular  assignment  of  memory  planes  into  which 
faces  are  written  determines  their  relative  occul¬ 
tation  priorities.  During  the  read  cycle,  these 
memory  plane  assignments  are  used  to  resolve  the 
correct  output  color  for  overlapping  objects. 

It  should  be  noted  that  although  the  vector 
generator  removes  potential  streaks  by  deleting 
some  face  edge  points,  a  separate  set  of  memory 
planes  is  utilized  to  insure  correct  definition 
of  face  boundaries.  During  the  read  cycle,  this 
Information  is  used  in  conjunction  with  the 
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smoothing  factor  to  determine  the  correct  color 
of  pixels  representing  face  outlines. 

The  smoothing  factors  are  derived  by  table 
lookups  for  each  pixel  as  it  is  traversed.  The 
method  of  implementation  insures  that  valid  re¬ 
sults  are  obtained  for  all  possible  face  edge 
crossings  within  a  given  pixel.  The  resultant 
smoothing  factor  represents  the  area  subtended 
by  the  highest  priority  face  or  group  of  faces 
passing  through  that  pixel.  This  factor  is 
used  for  the  color  mixing  ratio  during  the 
read  cycle  operations. 

During  the  ready  cycle,  the  digital  scene 
image  in  refresh  memory  is  converted  into  ana¬ 
log  video.  During  the  write  cycle  the  refresh 
memory  is  addressed  by  the  vector  generator  in 
face  edge  outline  order.  In  this  cycle,  however, 
the  master  timing  generator  addressed  refresh 
memory  in  raster  scan  order,  exactly  as  dis¬ 
played  on  the  CRT  monitor.  At  each  pixel,  a 
color  is  resolved  that  is  either  the  color  of  an 
individual  face  (no  edge  exists  -  face  interior) 
or  a  color  mixture  (edge  exists  -  face 
boundaries). 

The  read  cycle  operations  ‘nclude: 

•  Face  color  fill  ng 

•  Face  color  resolution  (occulation) 

•  Face  edge  srv-  thing 

Face  color  fill  it- ■  is  the  mechanism  which 
fills  the  area  bound*-.,  iv  the  face  edge  outline 
by  turning  on  a!1  pixels  between  left  and  right 
side  face  edges  along  each  raster  scan  line.  The 
filling  operation  is  executed  only  on  memory 
planes  dedicated  to  ‘.torage/retrieval  of  face 
outline  data,  not  ca  those  designated  to  highest 
priority  edges  or  smoothing  information. 

Color  resolution  is  the  process  of  determin¬ 
ing  the  output  color  of  each  individual  pixel. 

For  those  pixels  containing  no  edge  data,  the 
resolved  output  is  simply  the  highest  priority 
fill  information.  For  pixels  containing  edges, 
additional  logic  tests  are  performed  to  resolve 
the  relative  priorities  of  filling  information 
versus  information  existing  on  those  planes 
designated  to  store/ retrieve  the  highest  prior¬ 
ity  edges.  As  a  result  of  this,  multiple  memory 
plane  outputs  are  determined. 

For  pixels  containing  fill  information  only, 
the  resolved  output  is  used,  as  an  address  into  a 
video  lookup  table  (VLT) .  Pixels  containing  edge 
data  require  multiple  simultaneous  VLT  accesses. 
The  output,  in  either  case,  is  a  digital  color 
code  representing  the  video  level  for  each  of  the 
three  primary  colors  (RED-GREEN- BLUE).  If  no 
edge  data  exists,  the  color  code  is  converted 
directly  into  analog  video  (RGB)  via  digital  to 
analog  conveters.  If  edge  data  does  exist,  color 
codes  are  mixed  according  to  the  smoothing  factors 
prior  to  the  conversion.  The  mixing  is  performed 
as  a  weighted  average  of  the  subpixel  areas  sub¬ 
tended  by  the  faces. 


ILLUMINATION  CONTROL 

The  illumination  control  hardware  provides  the 
final  image  processing  function  prior  to  the  dis¬ 
play  of  the  scene  on  the  CRT  monitor.  Utilizing 
analog  video  inputs  from  the  display  generation 
and  control  parameters  originating  in  the  general 
purpose  computer,  this  device  enhances  the  degree 
of  scene  realism  by  incorporating  various  visi¬ 
bility  effects  into  the  video  before  it  is  finally 
sent  to  the  monitor. 

The  illumination  control  unit  is  an  analog/ 
digital  hybrid  device  capable  of  generating  a 
variety  of  atmospheric  and  illumination  effects. 
These  effects  include  simulation  of: 

•  Horizon  glow 

•  Haze/Fog 

•  Smoke 

•  Curved  surface  shading 

Control  data  for  these  effects  are  transferred 
to  the  device  during  the  raster  vertical  blanking 
period.  During  the  display  period,  these  parame¬ 
ters,  along  with  timing  and  enabling  signals  from 
the  display  generator,  are  used  to  modifiy  the  RGB 
video. 


HARDWARE 

The  GVS-1  hardware  resides  in  two  standard 
equipment  enclosures.  Cabinet  1  contains  the 
general  purpose  computer,  its  peripheral,  and  the 
geometric  processor.  Cabinet  2  contains  the  dis¬ 
play  generator  and  illumination  control,  with 
sufficient  space  remaining  to  allow  additional 
hardware  to  be  added  to  meet  special  customer 
requirements  such  as  texture  generation. 


CONCLUSIONS 


The  GVS-1  is  a  new,  cost-effective  CGI  system 
that  develops  realistic,  full  color  images.  The 
construction  is  simple,  using  less  components  than 
conventional  CGI  systems  of  similiar  quality.  The 
system  is  modular  in  design  allowing  quantitative 
expansion  of  the  number  of  edges  and/or  instan¬ 
taneous  colors,  and  qualitative  improvements  in 
the  area  of  edge  smoothing. 


In  addition,  the  system  is  capable  of  accepting 
the  output  of  texture  generators  which  increase 
the  realism  of  the  scene  image.  The  texturing 
capability  of  the  Gould  Visual  System  has  been 
successfully  demonstrated  for  an  observer  having 
full  freedom  of  motion  in  azimuth  and  elevation. 

In  this  case,  texture  has  been  used  on  edge- 
defined  faces  and  independently  to  develop  complex 
texture  objects  (e.g.  pine  tree  forest).  Texture 
objects  have  full  occulting  power  similar  to 
terrain  and  moving  models. 
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Since  these  texture  patterns  are  generated 
and  stored  offline  and  read  out  in  real  time  from 
a  special  set  of  memory  planes,  they  do  not 
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ABSTRACT 


Computer  image  generation  (CIG)  successfully  provides  images  for  a  large  proportion  of 
visual  simulation  tasks,  using  polygon  models  of  the  environment.  However,  for  close  ap¬ 
proach  to  terrain,  e.g.,  for  air/ground  weapon  delivery,  confined  area  maneuvering  and 
harbor  navigation,  the  image  detail  is  insufficient  and  the  cost  of  generating  the  high 
detail  data  bases  is  becoming  increasingly  prohibitive. 

The  authors  are  developing  a  new  form  of  CIG  -  the  Real  Environment  Algorithm  for  Line  Scan¬ 
ning  (Realscan)  System,  using  a  uniform  grid  type  digital  data  base  of  contoured,  textured 
terrain  semi -automatical ly  generated  from  Defense  Mapping  Agency  (DMA)  and  stereo  photo 
data.  The  system  will  use  video  disc  bulk  storage,  potentially  visible  data  (for  a  given 
position  and  attitude  of  the  simulated  vehicle)  being  read  into  random  access  memory  and 
then  addressed  in  accordance  with  an  algorithm  giving  correct  perspective  mapping  into  the 
display  in  real  time  with  automatic  elimination  of  hidden  areas  of  terrain  and  without 
aliasing  (sampling  artifacts). 

The  image  detail  in  a  system  of  this  type  can  be  limited  only  by  the  display  resolution, 
giving  a  very  large  increase  in  scene  detail  over  what  is  likely  to  be  available  for 
many  years  with  polygon  modeled  CIG.  The  application  of  the  system  to  IR  is  being 
studied  with  the  aim  of  producing  coordinated  visual /IR/ radar  displays  generated  from 
a  common  data  base. 


INTRODUCTION 

One  of  the  outstanding  problems  of  visual  simu¬ 
lation  is  the  generation  of  a  complex  scene.  Al¬ 
though  satisfactory  training  can  be  achieved  for 
many  simulation  tasks  using  current  computer  Image 
generation  (CIG)  systems  at  their  present  stage 
of  development,  close  approach  to  terrain  in  day¬ 
light  reveals  the  severe  limitations  for  tasks 
such  as  air/ground  weapon  delivery,  confined 
area  maneuvering,  low  level  flight  and  harbor/ 
channel  navigation  with  both  surface  vessels  and 
submarines. 

With  conventional  CIG,  the  environment  is  modeled 
by  defining  the  coordinates  of  points  in  space, 
grouping  pu'.rs  of  points  to  define  "edges",  con¬ 
necting  edges  to  form  polygons,  assigning  color 
and  reflectivity  to  the  polygons  and  disposing 
them  to  form  polyhedra  which  approximate  the 
environment  to  a  level  limited  by  the  processing 
power  of  the  real  time  CIG  hardware.  The  number 
of  edges  processed  per  displayed  scene  is  the 
usual  metric  for  such  hardware  with  the  state-of- 
the-art  currently  around  8,000. 

For  modeling  regular  objects  such  as  a  runway. 


polygon  modeling  is  economical  in  terms  of  the 
size  of  data  base  and  the  amount  of  processing 
hardware.  However,  a  complex  environment  such  as 
an  area  of  countryside  consisting  of  contoured 
terrain  having  detailed  surface  texture  and  solid 
objects  such  as  trees,  building,  etc.  on  it,  is 
not  realistically  reproduced  by  present  CIG  sys¬ 
tems  for  two  reasons:  modeling  cost  and  proces¬ 
sing  hardware  limitations  of  the  polygon  modeling 
technique.  Further  development  in  polygon  model 
type  CIG  processing  hardware  is  in  progress  by 
several  manufacturers  (the  "edge  war")  but  the 
limiting  factor  is  not  so  much  the  real  time  pro¬ 
cessing  hardware  as  the  cost  of  generating  the 
increasingly  complex  environment  models  or  data 
bases  required. 

AUTOMATIC  DATA  BASE  GENERATION  PROBLEMS 

Recent  studies  (1)  at  the  Naval  Training  Equip¬ 
ment  Center  (NAVTRAEQUIPCEN)  on  the  automatic 
generation  of  polygon  type  data  bases  from  stereo 
photographs  showed  promise  for  individual  objects 
but  underlined  the  great  difficulty  of  the  prob¬ 
lem  for  terrain.  Automating  the  production  of 
polygon  models  of  real  world  environments  is  a 
task  being  addressed  by  the  operational  equipment 
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community  specifically  with  regard  to  Cruise 
Missile  guidance  systems,  and  success  is  not  ex¬ 
pected  for  many  years. 

The  reason  for  the  difficulty  can  be  appreciated 
if  one  looks  at  almost  any  aerial  photograph  and 
mentally  tries  to  break  it  down  into  separate 
objects  that  can  be  approximated  by  polygons. 

Much  of  what  is  visible  can  only  be  recognized 
by  a  skilled  photo-reconnaisance  technician.  The 
difficulty,  therefore,  is  fundamental:  the  pro¬ 
cess,  to  be  automated,  requires  a  degree  of 
artificial  intelligence  greater  (for  this  task) 
than  that  of  the  average  human  being.  This 
basic  difficulty  emphasizes  the  need  for  consi¬ 
dering  the  entire  image  generation  system  from 
data  base  modeling  to  image  display  when  develop¬ 
ing  an  advanced  CI6  concept,  and  for  evaluating 
non-polygon  type  data  bases  as  a  way  out  of  the 
automatic  data  base  generation  problem. 

OUTLINE  REQUIREMENTS  FOR  NEW  SYSTEM 

NAVTRAEQUIPCEN  is  currently  pursuing  an  explora¬ 
tory  development  program  for  an  advanced  CIG 
system.  The  aims  of  the  development  are: 

a.  Produce  highly  complex  terrain  scenes 
with  detail  approaching  the  limit  of  a  unique 
scene  element  for  each  image  display  picture 
element. 

b.  Allow  direct  and  automatic  conversion 
of  real  world  height,  color  and  reflectivity 
information  into  the  CIG  data  base. 

c.  Provide  for  large  and  variable  gaming 
areas  from  which  true  perspective  scenes  can  be 
computed  and  displayed  in  real  time. 

d.  Provide  compatibility  of  the  new  system 
with  polygon  model  type  CIG  for  optimum  flexi¬ 
bility  and  economy  in  defining  a  total  image 
generation  and  display  system. 

DEVELOPMENT  OF  NEW  SYSTEM  CONCEPT 

If  one  attacks  the  requirements  individually  the 
following  system  characteristics  emerge.  First 
of  all,  high  scene  detail  requires  a  substantial 
data  base;  however,  the  storage  requirements  are 
less  if  two  of  the  ground  coordinates  are  ad¬ 
dresses  rather  than  data.  Large  areas  of  the 
world  are  mapped  as  uniform  grid  models,  such  as 
the  Terrain  File  of  the  Digital  Landmass  System 
(DLMS)  developed  by  the  The  Defense  Mapping 
Agency  and  the  ortho  color  photos  produced  by  the 
U.S.  Geological  Survey  (2).  Models  of  this  type 
are  relatively  easy  to  generate  by  low-skill  pro¬ 
cesses  (as  compared  with  polygon  models)  and  offer 
the  greatest  possibilities  for  use  In  a  CIG  sys¬ 
tem  of  the  type  we  are  considering. 

To  obtain  constant  angular  resolution  of  visible 
details  over  the  display,  the  corresponding 


ground  resolution  required  decreases  with  range 
from  the  viewpoint.  Since  a  change  of  viewpoint 
means  all  parts  of  the  data  base  will  at  some 
time  be  viewed  at  close  range,  the  data  represent 
ing  the  gaming  area  must  all  be  available  at  high 
resolution.  However,  for  any  given  viewpoint, 
to  avoid  having  to  process,  in  real-time,  high 
resolution  data  representing  distant  terrain,  a 
lower-detail  version  of  the  data  may  be  used  for 
distant  parts  of  the  scene.  This  leads  to  the 
concept  of  a  hierarchy  of  resolution  levels  in 
the  data  base,  analagous  to  the  levels  of  detail 
used  in  polygon  model  CIG. 

However,  whereas  the  levels  of  detail  in  a  poly¬ 
gon  model  data  base  must  all  be  created  by  the 
modeler,  with  a  regular  grid  data  base  each  level 
can  be  derived  automatically  off-line. 

In  current  real-time  CIG  systems,  following  trans 
fer  of  the  polygons  forming  the  scene  into  the 
display  plane,  sorting  algorithms  are  used  to 
produce  the  displayed  image  and  the  processing 
power  required  grows  proportionally  to  the  square 
of  the  detail  processed.  For  systems  dealing 
with  a  great  amount  oT  detail  in  real  time,  an 
architecture  is  needed  in  which  processing  power 
grows  only  linearly  with  detail,  and  this  should 
be  a  feature  of  the  new  system.  To  process  and 
display  a  great  amount  of  displayed  detail  in 
real  time  a  modular  structure  is  indicated  making 
use  of  basically  independent  parallel  processors, 
each  a  pipeline. 

The  use  of  a  ground  coordinate  grid  allows  incre¬ 
mental  processing  in  a  fixed  data  base.  This 
feature  eliminates  the  need  for  floating  point 
operations  and  repeated  need  for  divide  opera¬ 
tions.  Hence,  computationally  efficient  integer 
algorithms  can  be  developed  which  incorporate  a 
pseudo  exponentiation  upon  crossing  from  one 
level  of  the  data  base  to  the  next  (using  a  step 
of  2:1  in  linear  detail  between  levels). 

Finally,  the  generated  image  should  be  in  one  of 
the  standard  television  formats,  e.g.  1023  lines/ 
frame,  30  frames/sec.  This  allows  existing  dis¬ 
plays  to  be  used  and  combined  image  generation 
systems  to  be  developed  using  both  polygon  and 
grid  data  forms  of  CIG. 

DESCRIPTION  OF  SYSTEM 

REALSCAN  (Real  Environment  Algorithm  for  Line 
Scanning)  is  a  concept  being  developed  at 
NAVTRAEQUIPCEN  to  produce  highly  detailed  terrain 
displays  directly  from  a  uniform  square  grid  data 
base  in  real  time.  This  concept  utilizes  a  hier¬ 
archy  of  resolution  levels  to  model  the  environ¬ 
ment  and  many  parallel  -  pipelined  processors  to 
perform  the  required  visibility  determination 
followed  by  perspective  transformation  from 
world  coordinates  to  display  coordinates.  The 
processing  is  based  on  computationally  efficient 
Integer  algorithms. 
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Figure  1.  Oisplay  Window  Projection 


Figure  1  shows  the  projection  of  the  display  win¬ 
dow  on  to  the  ground  plane  to  give  the  "ground 
footprint"  from  which  data  must  be  transformed 
into  the  display  window  coordinates.  Each  level 
cf  the  data  base  consists  of  a  square  array  of 
"ceils",  each  cell  containing  digital  color  and 
height  data  for  that  elementary  area  of  the 
terrain  modeled.  Although  all  levels  of  the 
hierarchy  are  available  over  the  whole  gaming 
area,  level  1  is  used  for  parts  of  the  terrain 
nearest  to  the  eyepoint  and  levels  2,  3  etc., 
are  used  as  the  range  increases.  The  determina¬ 
tion  of  which  points  are  visible  is  made  by  ra¬ 
dially  scanning  the  ground  footprint  in  a  manner 
similar  to  that  used  in  radar  landmass  simulation. 
Each  radial  line  drawn  from  the  nadir  point  (the 
point  directly  below  the  eyepoint]  through  the 
ground  footprint  maps  into  a  line  across  the 
display  window.  As  explained  later,  a  frame 
buffer  is  used  to  store  the  information  to  be 
displayed  such  that  the  information  can  be  read 
out  in  accordance  with  any  standard  television 
scanning  pattern  for  display  (e.g.  with  horizon¬ 
tal  scan  lines). 

New  scan  lines  are  Initiated  through  the  data 
base  as  the  scanned  distance  from  the  nadir  grows, 
such  that  the  visible  part  of  the  data  base  is 
correctly  sampled.  It  is  necessary,  for  accept¬ 


able  image  quality,  to  avoid  aliasing  (interac¬ 
tion  between  the  regular  structure  of  the  digital 
data  base  and  the  regular  structure  of  the  scan¬ 
ning  pattern  which  generates  spurious  detail  in 
the  displayed  image).  The  information  in  each 
displayed  picture  element  (pixel)  must  be  de¬ 
rived  from  the  data  stored  in  several  adjacent 
cells,  in  accordance  with  a  weighting  algorithm 
that  varies  with  the  position  of  the  pixel  in 
the  window.  This  requires  averaging,  in  real 
time,  of  the  processed  data. 

Real  time  averaging  over  many  cells  is  imprac¬ 
ticable,  but  the  hierarchical  data  base  concept, 
by  providing  pre-averaged  information,  reduces 
this  problem  to  manageable  proportions.  Fil¬ 
tering  is  used  to  eliminate  any  remaining  un¬ 
wanted  image  components. 

In  transforming  data  from  the  ground  plane  into 
the  viewing  window,  a  perspective  transforma¬ 
tion  has  to  take  place.  Each  scan  line  through 
the  data  base  is  dealt  with  separately,  the 
spacing  between  image  details  along  the  line 
being  changed  to  allow  for  the  oblique  view  of 
the  terrain.  Details  which  would  be  hidden  by 
high  ground,  are  of  course  omitted  from  the 
display. 
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Figure  2.  Functional  Block  Diagram  of  Realscan  System 


Figure  2  is  a  functional  block  diagram  of  the 
proposed  system.  The  color  and  height  data  for 
the  complete  gaming  area  are  stored  in  bulk 
memory;  a  videodisc  is  preferred  in  view  of  the 
large  number  of  bits  that  can  be  stored  in  a 
compact  space  with  reasonably  rapid  access  (this 
will  be  considered  in  more  detail  later  in  rela¬ 
tion  to  the  generation  of  the  data  base).  The 
cost  of  videodisc  bulk  memory  storage  is  cur¬ 
rently  estimated  to  be  $0. 10/megabyte  as  compared 
to  magnetic  disc  storage  costs  of  $3/megabyte 
(3). 

The  control  computer  accepts  simulated  vehicle 
position  and  attitude  data  from  the  host  com¬ 
puter  and  is  used  to  control  the  flow  of  data 
to  the  Realscan  system.  For  each  television 
frame  of  picture  information,  the  following  pro¬ 
cessing  steps  need  to  be  carried  out: 

a.  Obtain  observer  view  point  location  and 
viewing  direction  (simulated  vehicle  location 
and  attitude)  from  the  host  computer. 

b.  Compute,  in  the  control  computer,  the 
viewing  pyramid  intercepts  with  the  ground  ref¬ 
erence  plane. 

c.  Use  linear  and  angular  rates  of  the 
simulated  vehicle  to  predict  the  potential  field 
of  view  for  future  television  frames. 

d.  Determine,  using  the  control  computer, 
which  blocks  of  data  representing  hierarchical 
areas  of  terrain  in  the  bulk  memory  are  to  be  ac¬ 
cessed  for  generating  the  required  scene,  given 
the  nadir  point  location  and  field  of  view. 


e.  Using  the  control  computer,  transfer 
these  blocks  of  data  containing  all  the  informa¬ 
tion  necessary  to  compute  a  display  of  the  cur¬ 
rent  field  of  view  plus  additional  information 
for  future  frames,  to  the  "cell  memory"  (a  high¬ 
speed,  virtually  addressed  random  access  memory). 
After  the  cell  memory  has  been  initially  loaded, 
there  is  always  information  available  to  gener¬ 
ate  the  next  frames.  The  cell  memory  is  then 
the  first  stage  of  a  pipeline  computational  pro¬ 
cess  leading  to  a  frame  generation.  After  the 
current  frame's  information  is  read  from  the 
cell  memory,  those  blocks  which  are  no  longer 

in  the  potential  field  of  view  are  overwritten 
by  new  information. 

f.  Scan  the  cell  memory  using  a  group  of 
identical  scan  line  data  processors.  Each  pro¬ 
cessor  calculates  the  elevation  and  reflectivi¬ 
ty  information  along  its  scan  line  using  the 
regular  grid  data  in  the  cell  memory.  The  radial 
lines  are  scanned  in  parallel  groups  to  minimize 
the  time  taken  for  this  process.  The  data  is 
assembled  in  the  scan  line  detail  memory,  data 
for  each  scan  line  being  separately  stored. 

g.  Transfer  the  data,  in  parallel  as 
needed,  from  the  scan  line  detail  memory  to  the 
visibility  processor  and  carry  out  visibility 
processing  on  each  line  of  data.  This  process 
determines  which  parts  of  the  data  from  the  cell 
memory  are  to  contribute  to  the  displayed  scene 
and  which  are  hidden  from  view.  The  algorithm 
used  to  determine  visibility  is  similar  to  that 
used  to  determine  radar  shadows  in  digital  radar 
landmass  simulation  systems  (4,  5),  suitably 
modified  for  the  hierarchical  data  structure. 
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At  this  stage  only  the  elevations  of  cell  data 
enter  the  computation;  cell  reflectivity/color 
information  is  carried  with  no  modification. 

h.  Accumulate,  in  the  segment  of  visible 
scene  buffer,  reflectivity/color  data  from  the 
visibility  processor  in  first-in  first-out 
format,  to  allow  for  timing  variability  in  the 
pipeline  up  to  this  point. 

i.  Pass  the  data  from  the  segment  of 
visible  scene  buffer  to  the  color  pre-filter 
processor  and  perform  initial  filtering  along 
the  lines  to  avoid  aliasing  while  correctly  pass¬ 
ing  sudden  luminance  changes  corresponding  to 
non-aliased  edges  in  the  scene. 

j.  Pass  the  data  from  the  color  prefilter 
processor  to  the  projection  processor  where  the 
reflectivity/color  data  is  perspectively  trans¬ 
formed  from  ground  coordinates  to  display  coor¬ 
dinates. 

k.  Pass  the  output  from  the  projection 
processor  to  the  scene  memory  buffer. 

l.  Perform  final  filtering  using  a  weighted 
average  processing  filter  to  generate  display 
pixels. 

m.  Pass  the  data  into  a  random  write/ 
serial  read  ping-pong  frame  buffer. 

n.  Read  the  frame  buffer  continuously  into 
the  display. 

FEASIBILITY  OF  PROPOSED  SYSTEM 

None  of  the  components  required  for  the  proposed 
system  is  particularly  unusual;  however,  like 
any  CIG  system  it  is  complex  and  will  take  con¬ 
siderable  effort  to  implement. 

The  use  of  a  videodisc  as  the  bulk  memory  has 
already  been  referred  to.  The  control  computer 
can  be  an  off  the  shelf  item.  The  cell  memory 
size  is  a  function  of  the  access  time  of  the  bulk 
memory;  for  example,  an  access  time  cf  500  msec. 

(a  design  goal  for  the  videodisc  system)  would 
require  the  storage  of  sufficient  data  to  pro¬ 
duce  the  current  television  frame  plus  29  future 
frames  (assuming  a  position  update  of  60Hz). 

The  amount  of  new  information  required  per  frame 
is  a  function  of  the  rate  of  change  of  position 
and  look  direction.  With  the  above  access  time; 
a  horizontal  field  of  view  of  50°;  a  velocity  of 
lOOn/second  (200  knots);  a  heading  change  rate 
of  50°/sec;  a  high  resolution  cell  size  of  0.1m 
(with  appropriate  blocks  from  other  resolution 
levels);  the  cell  memory  size  requirement  is 
approximately  1  megabyte  (assuming  6  bytes/cell). 
This  does  not  present  any  special  problem. 

The  scan  line  data  processors  need  special 
attention.  At  this  stage  of  the  processing  the 
desired  display  resolution  becomes  a  considera¬ 
tion  in  determining  how  many  radial  scan  lines 
must  interrogate  the  cell  memory.  In  order  to 
carry  out  a  feasibility  analysis,  a  display 
having  the  usual  50°  horizontal  by  36°  verti¬ 
cal  format  is  assumed,  with  one  million  pixels. 

The  angular  subtense  of  the  diagonal  defines 
the  worst  case  width  of  the  current  field  of 
view  in  world  coordinates;  in  this  case  60°. 


From  sampling  and  anti-aliasing  considerations, 
it  is  desirable  to  have  approximately  twice  as 
many  radial  scanned  lines  computed  as  there  are 
displayed  pixels  across  the  display  for  the 
worst  case  condition.  In  this  example  approx¬ 
imately  3,000  radial  scans  should  suffice. 

In  principle,  3000  identical  processors  could  be 
used,  all  operating  in  parallel  to  scan  the  ra¬ 
dial  lines.  However,  the  overall  throughput 
delay  of  the  complete  CIG  system  can  be  kept 
smaller  than  that  of  polygon-type  CIG  systems 
with  approximately  300  processors  each  scanning 
10  lines.  Study  is  needed  to  determine  the 
optimum  number. 

The  plan  being  followed  is  to  develop  the  process¬ 
ing  algorithms  for  computational  efficiency  and 
ease  of  implementation  in  modular  hardware  as  a 
prime  consideration.  The  algorithms  have  been 
coded  into  software  and  tested  in  non-real  time 
using  the  VAX/11/780  at  the  NAVTRAEQUIPCEN  Com¬ 
puter  Laboratory.  The  data  base  used  for  initial 
testing  consisted  of  analytic  function  generators 
which  simulated  a  uniform  grid  of  elevation  and 
reflectance  values.  The  extension  to  operation 
on  a  real  world  uniform  grid  terrain  model  for 
elevation  and  a  digitized  ortho-photo  of  the  same 
terrain  area  for  the  reflectivity  is  the  next 
step.  After  an  evaluation  of  the  non-real  time 
imagery  and  iterations  of  the  algorithm  formula¬ 
tion,  a  feasibility  analysis  of  a  hardware  design 
will  be  carried  out  for  potential  real  time 
implementation. 

THE  REALSCAN  DATA  BASE 

The  last  section  presented,  in  outline,  a  likely 
architecture  for  the  system.  However,  to  judge 
the  value  of  the  system  it  is  necessary  to  re¬ 
late  it  to  the  data  base  and  to  quantify  some  key 
parameters. 

The  desired  data  base  form  is  a  hierarchy  of  two- 
dimensional  arrays  of  elevation  and  reflectivi¬ 
ty/color.  Each  array  corresponds  to  the  entire 
gaming  area  at  a  specific  ground  resolution.  The 
size  of  the  data  base  is  a  function  of  the  desired 
ground  detail  size.  it*  size  S’  t'C  gaming  area, 
and  the  amount  er  >•  tc>  mation  at  each  ar¬ 

ray  location. 

For  example,  consider  a  gaming  area  of  100  km2, 
a  desired  ground  detail  size  of  0.1m,  and  48 
bits  of  information  stored  at  each  array  location. 
The  largest  array,  corresponding  to  the  highest 
resolution,  would  contain  1010  grid  positions  or 
cells  and  5x10^  bits  of  information.  If  the  cell 
size  in  each  array  is  doubled  or  the  resolution 
is  half  that  of  the  adjacent  array  In  the  hier¬ 
archy  then  the  total  data  base  required  for  the 
entire  hierarchy  is  (1  +  1/4  +  1/16  +  1/64  ...) 
times  the  information  stored  in  the  highest 
resolution  array,  or  approximately  7X1011  bits 
for  the  total  data  base.  Although  this  is  a 
large  number,  the  optical  disc  digital  storage 
technology  has  progressed  to  the  point  where 
such  large  data  bases  are  feasible.  In  fact, 
optical  disc  storage  configurations  have  been 
proposed  for  systems  having  a  capacity  of  1014 
bits  (6).  The  next  question  is:  What  is  the 
value  of  a  regular  grid  data  base  with  0.1m 
ground  resolution?  For  close  approach  to  the 
ground,  as  with  a  helicopter  maneuvering  in  a 
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confined  area  landing  site,  a  minimum  range  from 
the  pilot's  eye  to  ground  could  be  taken  as  5m. 

Considering  a  television  display  with  1000  hori¬ 
zontal  scanning  lines  and  50“  wide  field  of  view 
(a  typical  CI6  "viewing  window"),  the  ground 
resolution  (detail  size)  should  be  of  the  order 
of  —  iTojfli  or  >-2  cm.  This  is  smaller  than  the 
figure  of  0.1m  chosen  above  by  a  factor  of  20. 

To  provide  the  desired  fine  detail  in  the  dis¬ 
play,  the  basic  concept  is  to  interpolate  special 
functions. 

Example  of  data  base  construction 

a.  DLMS  terrain  file  (Level  I)  consists 
of  an  elevation  array  with  a  grid  spacing  of 
approximately  100m.  This  array  would  be  inter¬ 
polated  using  bicubic  functions  to  form  the  ini¬ 
tial  hierarchy  arrays.  In  the  case  of  a  desired 
0.1m  ground  detail  the  number  of  arrays  generated 
would  be  10  corresponding  to  cell  sizes  of  0.1, 

0.2,  0.4 .  25.6,  51.2m. 

b.  The  enhancement  of  the  elevation  models 
can  be  carried  out  in  a  variety  of  ways.  By 
making  use  of  the  DLMS  cultural  file,  generic 
cultural  features  which  have  been  stored  in  a 
feature  library  in  the  gridded  elevation  format 
at  the  various  resolution  levels  can  be  added  to 
(or  subtracted  from)  the  initial  elevation  data. 

The  generic  models  can  be  obtained  by  photogram- 
metric  techniques  using  stereo  photos  of  repre¬ 
sentative  real  world  cultural  features. 

c.  Reflectivity/cc  information  can  also 
be  generic  or  specific.  -.is  information  can  be 
derived  from  digitized/qjantized  color  ortho 
photos.  (An  ortho  photo  is  a  product  of  automatic 
analytic  stereo  photogrammetric  equipment  in  which 
the  image  displacement  due  to  relief  is  corrected). 
If  the  reflectivity/color  model  for  a  specific 
lOOknr  area  is  desired,  approximately  fifty  color 
aerial  photographs  would  be  -squired  to  obtain 

the  raw  reflectivity  photographic  data  to  0.1m 
cell  size.  If  the  generic  route  is  chosen,  appro¬ 
priate  color  orthos  photos  of  representative  cul¬ 
tural  features  are  utilized  to  form  a  libr  -y 
which  corresponds  to  the  generic  elevation  i- 
brary. 

d.  The  high  resolution  reflectivity  in¬ 
formation  is  interpolated  to  form  the  reflectivi¬ 
ty  data  for  succeeding  lower  levels  in  the  hier¬ 
archy.  In  this  way  scene  compatibility  across 
hierai  hi  :al  levels  is  ensured. 

e.  In  addition,  elevation  and  reflecti¬ 
vity/color  function  codes  can  be  assigned  to  each 
cell.  These  codes  describe  the  fine  detail  re¬ 
ferred  to  in  b.  above  and  provide  for  blending  of 
reflectivity/color  and  terrain  height  between 
cells  so  as  to  avoid  discontinuities.  This  scheme 
enables  the  aim  of  making  the  system  generate  as 
much  detail  as  can  be  displayed  to  be  achieved 
without  excessive  memory,  although  individual 
objects  cannot  be  dealt  with  in  this  way.  Fur¬ 
ther  elaboration  to  the  system  shown  In  Figure  2 
is,  of  course,  required. 

The  net  result  is  a  highly  automated  data  base 
modeling  system  which  is  not  dependent  on  sub¬ 
jective  decisions  by  the  modeler  and  can  be 
Implemented  using  available  data  and  technologies 


in  the  photogrammetric  conmunity. 

The  concept  of  essentially  overlaying  a  digital 
elevation  model  with  photographically  derived 
reflectivity  data  to  form  an  environment  model 
is  not  new  (7)  (8)  (9),  but  the  use  of  such  a 
model  in  a  real  time  C16  system  has  not  been  at¬ 
tempted  to  our  knowledge. 

The  reasons  for  pursuing  this  approach  to  envir¬ 
onment  modeling  are  summarized  as: 

a.  The  need  for  efficiently  produced  low 
cost,  highly  detailed  envirorment  models. 

b.  The  availability  of  equipments  and 
technologies  which  can  produce  highly  detailed 
environment  models  in  a  uniform  grid  format. 

c.  The  availability  of  technology  to  store 
and  access  large  amounts  of  data. 

d.  The  feasibility  of  an  automatically 
generated  level  of  detail  hierarchy  which  great¬ 
ly  reduces  computational  processing  load,  since 
simulated  long  ranges  utilize  coarser  levels  of 
the  hierarchy. 

The  environment  model  described  has  been  ini¬ 
tially  restricted  to  terrain  surfaces  which  are 
single  valued  in  elevation,  and  the  acceptability 
for  training  using  such  a  model  has  not  been 
evaluated.  However,  the  potential  for  extension 
to  multivalued  elevation  and  vertical  surfaces 
has  been  considered  and  various  algorithms  for 
implementation  are  being  evaluated  to  represent 
multi-valued  elevation  features  such  as  clouds 
(10). 

THE  USE  OF  THE  REALSCAN  CONCEPT  IN  SIMULATION 

As  already  indicated,  the  proposed  system  can 
generate  a  perspective  scene  in  any  television 
format  and  so  is  compatible  with  polygon  model 
CIG.  In  fact,  its  likely  eventual  use  would  be 
in  a  visual  system  having  a  Real  scan-generated 
contoured  textured  terrain  with  polygon-modeled 
target  vehicles  and  buildings. 

The  generation  of  irregular  objects  such  as  trees 
remains  a  problem  for  polygon  CIG  although  vari¬ 
ous  approaches  are  being  investigated.  An  adap¬ 
tation  of  Real  scan  may  be  of  value  here,  using 
automatic  generators  of  a  digital  model  from  a 
physical  model . 

Just  as  polygon  CIG  image  generators  are  built  in 
channels,  each  channel  feeding  a  display  window, 
so  can  Realscan  be  built  in  a  similar  way.  Wide 
angle  displays  of  multiple  windows  are  equally 
feasible. 

Particularly  interesting  are  the  possibilities 
for  the  Realscan  concept  in  association  with  cur¬ 
rent  display  systems  under  development  in  which 
the  pilot  is  presented  with  a  display  over  a 
limited  field  of  view  in  the  direction  in  which 
he  is  looking,  using  head  and  eye  tracking  to 
control  the  image  generator.  Such  a  combination 
of  image  generator  and  display  would  give  very 
high  effective  detail  to  the  pilot  for  any  posi¬ 
tion  of  his  head  and  eyes. 

Finally,  the  increasing  use  of  Infrared  sensors 
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must  be  mentioned.  The  simulation  of  an  IR  sys¬ 
tem  requires  a  data  base  even  more  demanding  for 
some  systems,  than  visual  simulation  and  the  IR 
and  visual  data  bases  must  possess  the  same  cor¬ 
related  geometry  as  the  radar  data  base.  A 
Realscan  type  system  may  in  the  long  run  unify 
the  simulation  of  all  these  sensors. 
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ABSTRACT 

A  conceptual  design  is  presented  of  a  large  field-of-view.  high  resolution  visual  system  with  an  integrated 
flexible  cab  configuration  that  can  be  procured  with  a  high  degree  of  confidence  in  the  1982-84  time  period.  The 
mission  requirements  are  defined  for  the  Army  Rotorcraft  System  Integration  Simulator  (RSIS)  which 
incorporates  this  Advanced  Cab  and  Visual  System  (ACAVS).  A  brief  description  is  provided  of  the  NASA-Ames 
Vertical  Motion  Simulator  as  it  will  be  configured  with  ACAVS  installed.  Major  ACAVS  system  requirements  are 
addressed  and  some  attention  is  given  to  their  relationships  to  the  intended  mission.  Four  existing  visual  display 
technologies  and  computer  generated  imagery  approaches  are  identified  and  their  potential  application  to 
ACAVS  is  described.  The  ACAVS  conceptual  design  is  presented  and  a  comparison  is  made  of  major 
requirements  and  goals  to  final  system  specifications 

The  paper  closes  with  a  brief  discussion  of  potential  applications  of  the  RSIS  to  future  helicopter  systems 
design,  integration,  product  improvement  evaluations  and  safety  analysis. 


INTRODUCTION 

Over  the  last  20  years,  flight  simulators  have  emerged  as  a 
recognized  and  widely  accepted  training  tool.  A  survey  of  all  23 
scheduled  U  S  Airlines  revealed  that  more  than  70  modern  flight 
simulators  are  owned  and  operated  by  the  1 8  airlines  who  replied  to 
the  survey  Currently,  a  very  large  part  cf  all  commercial  airline  flight 
training  is  conducted  in  ttiose  flight  simulators1'1 

The  use  of  simulators  for  training  within  the  Department  of 
Defense  (DOD)  began  during  World  War  II  to  increase  instrument 
flying  proficiency  During  the  next  twenty  years,  the  feasibility  of  real 
time  digital  simulators  for  flight  training  was  demonstrated  Weapons 
systems  simulators  followed  with  the  development  of  the  terrain 
model  board  and  TV  approach  for  visual  simulation.  In  1969  Apollo  tl 
landed  on  the  moon  validating,  on  the  first  flight,  the  effective  use  of 
total  simulation  in  the  training  program121 

While  their  value  has  been  widely  exploited  by  the  fixed-wing 
industry  for  many  years,  piloted  flight  simulators  have  seen  much 
less  use  by  the  rotary-wing  industry.  In  1971 .  the  U  S.  Army  initiated 
an  extensive  program  in  the  use  of  simulators  for  training  helicopter 
aircrews  with  its  introduction  of  the  UH-1H  Synthetic  Flight  Training 
System.  Since  then  training  simulators  have  been  developed  for  the 
CH-47  Chinook  and  AH-1  Cobra,  and  one  is  undergoing  acceptance 
testing  for  the  UH-60A  Blackhawk  Similarly,  the  U  S.  Navy 
introduced  a  Weapons  System  Trainer  for  the  SH-2F  Seasprite  in 
1976  and  have  systems  under  development  for  the  CH-46E  Sea 
Knight  and  SH-3H  Sea  King131 

In  the  fixed-wing  aircraft  industry  the  cost-effectiveness  of  piloted 
flight  simulators  has  also  been  demonstrated  in  research  and 
development'’1'.  It  has  become  a  primary  tool  in  the  understanding  of 
the  flight  characteristics  of  new  aircraft,  the  development  of 
certification  criteria,  the  validation  of  aircraft  control  concepts,  and 
the  formulation  of  new  approaches  to  air  traffic  control  procedures. 

In  contrast,  there  has  been  limited  exploitation  of  man-in-the-loop 
simulation  during  the  research  and  development  phases  of 
rotary-wing  aircraft.  Some  examples  are  the  use  of  Northrop' s 
capabilities  during  development  of  the  Heavy  Lift  Helicopter, 
simulations  performed  by  Sikorsky  and  NASA-Langley  in  supporl  of 
the  tilt  rotor  development  and  Stability  and  Control  Augmentation 
System  (SCAS)  failure  investigations  on  the  Bell  214 

In  1975,  a  joint  U  S.  Army  and  NASA  study  was  performed  to 
review  the  functions,  status  and  future  needs  for  ground-based  flight 
simulation  of  rotary-wing  aircraft.  Contacts  were  made  with  the  U  S 
helicopter  industry  and  with  (he  various  U  S  agencies  concerned 


with  the  development  of  rotary-wing  systems  to  assess  the  needs  for 
research  simulation.  In  the  course  of  this  review,  the  deficiencies  in 
current  simulation  capability  relative  to  rotary-wing  aircraft 
requirements  were  defined  with  consideration  of  all  the  special 
aspects  of  this  problem  including  mission,  tasks,  aircraft 
characteristics,  environmental  conditions,  instrumentation  and 
displays,  performance  and  workload  Many  of  these  aspects  impose 
requirements  quite  different  from  those  met  by  even  the  most 
sophisticated  fixed-wing  piloted  simulators  As  a  result  of  this 
review'61,  a  program  was  initiated  to  develop  a  high-fidelity  rotorcraft 
srmulation  capability  tnat  could  be  exploited  by  both  government  anti 
industry  in  Research  and  Development  (R&D)  This  simulation 
capability  is  being  developed  jointly  by  the  Aeromechanics 
Laboratory  and  NASA  at  NASA-Ames  Research  Center.  Moffett 
Field.  California.  The  Aeromechanics  Laboratory  is  one  of  four 
research  laboratories  assigned  to  the  Research  and  Technology 
Laboratories  (RTL)  of  the  U  S  Army  Aviation  Research  and 
Development  Command  (AVRADCOM) 

Research  Simulator  Requirements 

There  are  many  differences  between  fixed-wing  aircraft  and 
rotary-wing  aircraft  that  imply  different  simulation  requirements 
Generally,  fixed-wing  aircraft  fly  high  and  relatively  fast  and  are  close 
to  the  ground  only  when  landing  or  when  taking  off.  In  contrast, 
helicopters  fly  low  and  slow  and.  especially  during  military  missions, 
are  in  close  proximity  to  the  ground  during  most  of  their  flying  time 
The  term  Nap-of-the-Earth  (NOE)  (Figure  t)  has  been  coined  by  the 


■mud*. 

NOE  ®  CONTOUR  (§)  LOW  LEVEL  © 


Figure  1.  Terrain  Flying  Regime* 

♦Much  of  the  information  presented  in  this  paper  was  developed  on 
NASA  Contract  NAS2-10464,  NASA-Ames  Research  Center,  Moffet 
Field.  CA 
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helicopter  community  to  describe  operations  in  which  they  fly  only  a 
few  feet  above  the  ground  and  fly  around  obstacles  rather  than  over 
them.  The  environment  for  the  pilots  flying  these  missions  is  rich  in 
detail  -  full  of  trees  and  bushes,  hills,  and  valleys,  which,  while 
offering  protection  from  the  enemy,  are  lethal  to  an  unwary  pilof. 
Terrain  features,  visibility  factors  of  weather  and  darkness,  and 
atmospheric  characteristics  of  wind,  turbulence,  and  ground  effect 
are  all  elements  of  the  environment  that  may  significantly  affect  the 
helicopter  pilot  s  tasks  The  helicopter  crew  must  maneuver  around 
and  between  obstacles  and  navigate,  communicate,  and  proceed 
with  the  mission  while  maintaining  awareness  of  threat  weapons. 

Fundamental  differences  in  the  environmental  cues  which  have  to 
be  simulated  for  rotorcraft  operations  compared  with  fixed-wing 
aircraft  result  from  these  different  flight  conditions.  The  visual  display 
is  required  to  represent  much  more  detail  in  the  terrain  and 
vegetation.  Being  close  to  the  ground,  terrain,  and  vegetation 
characteristics  cause  more  complex  atmospheric  wind  shear  and 
turbulence  characteristics.  Slow  flight  speed  and.  in  particular,  the 
conditions  in  and  around  hover  make  assumptions  of  a 
non-time-varying  turbulence  model  invalid,  and  satisfactory 
turbulence  models  more  difficult  to  achieve.  Slow  flight  and  high 
maneuverability,  especially  of  military  helicopters,  allow  rapid 
changes  of  flight  path  to  be  achieved  This  means  that  the  field  of 
view  required  for  the  pilot  to  see  where  he  is  going  is  wider  than  in  a 
fixed-wing  aircraft. 

The  basic  control  problems  are  more  difficult  too  In  a  fixed-wing 
aircraft  with  good  handling  qualities,  the  aircraft  is  stable,  and  control 
is  largely  a  two-axis  task  with  pitch  and  bank  angles  being  used  to 
direct  the  aircraft  flight  path.  In  a  helicopter,  especially  at  speeds 
approaching  hover,  pitch  attitude  becomes  less  effective  in 
controlling  flight  path  angles  and  more  effective  in  controlling  of 
speed,  while  an  additional  control,  thrust,  is  required  for  rate  of  climb. 
In  addition,  heading  is  no  longer  controlled  by  bank  angle  but  also 
requires  an  additional  specific  control  through  the  yaw  control  Thus, 
the  pilot's  control  problem  becomes  much  more  complex;  it  now 
requires  all  four  controls  to  be  actively  worked  In  addition  to  this 
fundamental  control  problem,  the  basic  helicopter  is  likely  lo  be 
unstable  and  to  have  significant  cross  coupling  between  the  various 
axes.  All  fhese  compfications  associated  with  helicopters  make  the 
need  for  the  additional  cues  provided  by  a  simulator  motion  system 
greater  than  in  the  case  of  fixed-wing  aircraft 

Finally,  the  mathematical  model  required  for  a  reasonable 
representation  of  a  helicopter  must  contain  some  elements  of  rotor 
dynamics,  the  extent  depending  on  the  purpose  and  nature  of  the 
simulation.  Thus,  the  requirements  on  the  visual,  motion,  and 
computational  aspects  are  all  different,  and  generally  significantly 
more  severe  than  those  for  a  simulation  of  similar  fidelity  for  a 
fixed-wing  aircraft. 

Project  Plan 

The  joint  Army/NASA  program  to  develop  a  high-fidelity  R&D 
simulator,  known  as  the  Rotorcraft  System  Integration  Simulator 
(RSIS),  is  now  in  its  final  phase.  The  first,  or  definition  phase,  started 
with  the  Army/NASA  study  in  1 975<5',  which  led  to  additional  studies 
to  address  the  issues  raised  by  the  different  requirements  discussed 
previously  for  rotorcraft  RAO  simulation  as  opposed  to  fixed  wing 
simulation.  A  feasibility  study  for  helicopterVTOL  wide-angle 
simulator  image  generation  displays  system  was  completed  by 
Northrop  in  1977,,).  Results  showed  that  wide  field  of  view  visual 
display  (1201-1  x  60°V)  was  feasible  using  the  techniques  of  image 
generation  by  camera  model  and  Image-presentation  by  three  color 
projectors.  The  image  presentation  technique  is  still  a  possible 
candidate  but  the  limited  resolution  and  inability  to  expand  the  field  of 
view  suggest  that  Computer  Generated  Imagery  (CGI)  may  be  a 
better  solution  for  image  generation.  Analysis  of  fixed-based 
simulations  of  NOE  flight  operations1 7’.  has  defined  the  cab 
excursions  required  for  high  fidelity  simulation  motion  Due  to 
budgetary  constraints  and  a  desire  to  make  maximum  use  of  existing 
facilities,  it  was  determined  that  the  Vertical  Motion  Simulator  (VMS) 
at  NASA-Ames  could  be  used,  with  modifications  at  the  motion  base 


for  the  RSIS.  The  VMS  will  be  described  in  more  detail  later  in  this 
report.  Two  independent  design  studies  to  assess  the  possible 
modification  were  performed  by  Franklin  Research  Laboratory  and 
Northrop  Corporation  in  1978,8M9>.  Specifications  were  developed 
from  these  studies  and  a  competitive  RFP  was  issued  to  industry  and 
the  contract  was  awarded  to  Franklin  Laboratory  in  1979 

A  new  interchangeable  rotorcraft  cab.  a  development  station,  and 
an  advanced  visual  system,  known  as  the  Advanced  Cab  and  Visual 
Systems  (ACAVS)  will  complete  the  RSIS  project.  The  development 
station  and  interchangeable  rotorcraft  cab  will  enable  Army/NASA 
researchers  to  release  the  VMS  for  additional  experiments  while 
reconfiguring  the  cab  in  the  development  station  for  the  next 
rotorcraft  experiment.  The  advanced  visual  system  will  be  a  wide 
field  of  view  system  with  look-down  capability.  An  overview  of  the 
RSIS  project  and  its  relationship  with  the  VMS  is  shown  in  Figure  2. 
The  final  configuration  with  all  component  parts  is  also  shown, 
conceptually,  in  Figure  3.  The  dome  concept  is  shown  as  a  feasible 
concept  only  and  does  not  imply  that  other  concepts  will  not  be 
considered  for  the  advanced  visual  system. 


RSIS  Project  Overview 


To  provide  an  effective  review  of  rapidly  advancing  technology, 
especially  in  the  visual  area,  a  preliminary  design  study  contract  was 
awarded  to  Boeing  Military  Airplane  Company  in  late  1979.  Their 
finding  will  be  discussed  next.  A  Statement  of  Work  and  RFP  was 
developed  from  their  study  and  is  awaiting  release  to  industry.  The 
ACAVS  contract  will  be  awarded  in  1981  and  the  RSIS  is  scheduled 
to  become  operational  in  1985. 
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VERTICAL  MOTION  SIMULATOR  DESCRIPTION 

The  Vertical  Motion  Simulator  (VMS)  is  a  large  man-carrying 
simulator  now  in  operation  at  Ames  Research  Center.  The  VMS 
consists  of  a  hydraulic  motion  system  mounted  on  a  structure  with 
large  lateral  and  vertical  motion  capabilities.  Vertical  motion  is  the 
primary  degree-of-freedom  and  all  other  modes  are  built  on  top  of  it. 
A  long  horizontal  platform  is  supported  by  two  vertical  columns. 
Currently  eight  (8)  DC  servo-motors  drive  the  simulator  sixty  feet 
vertically  through  gear  reducers,  pinions,  and  racks  which  are 
attached  to  the  columns.  Lateral  motion  capability  of  forty  feet  is 
provided  by  a  carriage  which  is  driven  across  the  vertical  motion 
platform.  Four  (4)  OC  servo-motors  drive  through  reducers  and 
pinions  to  engage  a  fixed  rack  on  the  vertical  motion  platform. 


As  a  part  of  the  RSIS  program  the  hydraulic  motion  system 
presently  mounted  on  the  vertical  and  lateral  structure  is  being 
replaced  with  the  Rotorcraft  System  Motion  Generator  (RSMG).  The 
overall  performance  envelope  of  the  combined  device  is  projected  to 
be  as  follows: 

Mode 

Displacement  Velocity 

Acceleration 

Vertical  (Z) 

»  30  ft.  ±20  ftsec 

£  32.2  ft  sec' 

Lateral  (Y) 

£  20  ft.  ±10  ftsec 

±  24  ft/sec2 

Longitudinal  (X) 

±  4  ft.  ±4  ftsec 

£  10  ft  sec2 

Roll 

»  78“  t  40°  sec 

±  115°/sec2 

Pitch 

£  18“  £  40° sec 

±  115°sec2 

Yaw 

t  24”  ±  46'  sec 

±  1 1 5°/sec2 

These  peak  motion  system  requirements  are  defined  for  a 
maximum  growth  payload  that  includes  all  hardware  attached  to  the 
motion  system  with  the  following  characteristics: 

Weight 

8,000  lbs 

Ix-x 

=  26,000  in-lb-sec2 

ly-y 

=  31.000  in-lb-sec2 

Iz-z 

=  31.000  in-lb-sec2 

ADVANCED  VISUAL  SYSTEM  REQUIREMENTS 


The  prime  projected  use  ol  RSIS  is  handling  qualities 
investigations.  In  rotorcraft  systems,  handling  qualities  depend 
heavily  upon  the  outside  visual  scene  for  most  missions. 
Requirements  for  ACAVS  visual  systems  stem  from  the  following 
NASA  user-defined  simulator  mission  tasks. 

Nap-of-The-Earth  Terrain  Flight 
Night  Operations 
Instrument  Flight 
Sling-Load  Control 
Conventional  Day  VFR 
Air  Combat 

These  tasks  require  a  visual  system  with  a  uniquely  wide  range  of 
performance.  The  most  stringent  requirements  flow  primarily  from 
the  NOE  mission  with  its  out-the-window  visual  scene  incorporating 
near  and  far  field  objects  that  are  viewable  over  a  significant 
percentage  of  full  field.  These  objects  represent  scenes  that  are  rich 
in  detail.  This  places  a  heavy  requirement  on  image  generation 
capability  particularly  when  realism  is  needed  for  pilot 
acceptance.1'01  For  the  purposes  of  ACAVS  preliminary  design  and 
development,  it  has  been  found  that  visual  display  and  CGI 
technologies  have  in  common  only  a  small  number  of  well  defined 
Interactions.  Primary  emphasis  in  this  paper  is  placed  on  the  display 
concept  as  the  visual  system  design  driver. 


For  clarity,  the  specific  requirements  associated  with  visual  display 
systems  and  those  associated  with  image  generation  will  be 
discussed  separately. 

Display  Systems  Requirements 

Specific  display  performance  requirements  that  were  used  in  the 
ACAVS  display  system  preliminary  design  are  shown  in  Table  t. 


Table  1.  ACAVS  Display  Performance  Requirements 


Parameter 

Minimum 

Goal 

Field  of  View 

120°H  x  60°V* 

240"H  x  180"V 

Image  Resolution 

6  arc  Min  .** 

3  arc  Min.** 

Bnghtness 

.03  30  IL 

03  50  fL 

Color 

2  Color 

Full  Color 

Contrast  Ratio 

.03  30 

- 

Slew  Rate 

60  “  Sec 

100°  Sec 

*  Instantaneous  FOV.  rotatable  as  a  goal 
**Measured  in  optical  line  pairs  as  seen  at  the  display 


Visual  parameters  most  critical  in  ACAVS  display  system  design 
are  Field  of  View  (FOV)  and  image  resolution  A  typical  view  polar  of 
a  large  side-by-side  rotorcraft  that  illustrates  the  demanding  FOV 
requirements  is  shown  in  Figure  4.  Other  parameters  of  importance 
in  the  overall  display  system  design  are  image  brightness,  color, 
contrast  ratio,  slew  rate  and  distortion.  A  full  discussion  of  the  origin 
of  these  performance  parameters  is  not  within  the  scope  of  this 
paper 


Design  Eyepoint 

The  FOV  requirements  include  a  60  downlook  potential  forward 
and  to  the  side.  A  sketch  of  the  percentage  of  full  field  view  is  shown 
in  Figure  5  for  both  the  minimum  requirement  and  the  desired  goal 

Resolution  requirements  are  driven  by  the  need  to  provide  precise 
control  of  linear  velocity  that  must  occur  near  the  terrain  and  to 
perform  target  tracking  tasks  that  may  include  acquisition  of  targets 
at  realistic  distances.  For  reference,  it  has  been  shown  that  detection 
ol  a  tank  at  2000  meters  requires  a  resolution  of  about  three  arc 
minutes  per  line  pair"21  Peripheral  vision  requires  somewhat  less 
resolution  since  the  relative  acuity  of  the  eye  drops  to  about  10%  at 
40°  from  the  center  of  the  fovea 

Visual  Image  Generation 

Visual  image  generation  is  an  important  and  complex  part  of  visual 
system  concept  design.  Even  though  the  major  thrust  in  this  paper  is 
directed  toward  display  concepts,  the  discussion  that  follows  offers 
an  abbreviated  overview  of  some  major  image  generation  design 
considerations. 
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Visual  Image  Generation 

There  are  many  factors  that  influence  image  generation  design 
These  include: 

Field  of  view  (FOV)  Image  resolution 

Scene  detail  Level  of  detail  control 

Number  of  displayable  edges  Moving  models 

Curved  surface  shading  Weapon  effects 

Color  Texturing 

Frame  rate  Atmospheric  effects 

Transport  delay  Gaming  area 

Three  of  these  factors  have  major  impact  on  the  design;  Field  of 
view,  image  resolution  and  scene  detail.  As  in  display  system  design, 
the  product  of  field  of  view  and  image  resolution  is  a  basic 
performance  measurement  and  relates  directly  to  the  number  of 
pixels  that  must  be  computed  in  computer  generated  image  systems 
(CGI).  Scene  detail  is  difficult  to  specify  in  such  a  manner  that  only 
those  requirements  are  placed  upon  (he  CGI  design  that  are  directly 
related  to  the  ACAVS  mission  task  This  is  an  important 
consideration  since  it  is  generally  accepted  that  any  attempt  to 
produce  individual  detail  equivalent  to  the  level  perceivable  by  the 
human  eye  would  be  futile  with  present  technology  However, 
texturing  methods  and  memory  intensive  algorithms  being 
developed  show  much  promise 


TECHNOLOGY  ASSESSMENT 

A  technical  assessment  of  systems  that  will  be  viable  for  ACAVS  in 
the  1982-84  time  period  indicates  advancement  of  several  visual 
system  components.  An  assessment  of  six  of  these  that  show 
particular  promise  is  given  here: 

1)  Television  projectors  are  nearing  production  that  use  liquid 
crystal  displays  with  significantly  lower  scene  lag  by  a  factor  of  at 
least  two  compared  with  those  previously  available. 

2)  Scanning  laser  systems  offer  high  resolution  images  over  a 
larger  field-of-view  per  projector  than  any  other  display  system 
surveyed 

3)  In-line  virtual  image  windows  are  being  made  lighter  weight  and 
more  light  efficient  by  substitution  of  holographic  elements  for  the 
heavy  optical  lenses 

4)  Studies  are  available  that  show  potential  of  expanding  the 
field -of-view  of  large  exit  pupil  virtual  imagery  systems  to  7CTV  x 
iam 

5)  High  resolution  fiber  optics  are  being  used  to  supply  images  for 
helmet  mounted  virtual  image  displays  or  for  real  image  projection  on 
large  spherical  screens 


6)  An  optical  extension  lens  display  technique  is  also  available  that 
will  lower  distortion  caused  by  off-center  projection  spherical  screen 
systems. 

Further  description  of  several  of  these  display  system  components 
is  provided  in  the  discussion  of  various  system  concepts. 

Image  Resolution  Factor 

A  factor  basic  to  the  information  presentation  ability  tor  wide  angle 
displays  generation  is  related  to  the  ratio  of  field-of-view  and 
image  resolution.  Because  of  the  interdependence  of  these 
parameters  their  product  may  be  used  and  form  a  figure  of  merit 
which  can  serve  as  a  comparison  between  potential  ACAVS 
presentation  sources  (e  g.,  television  projectors  and  scanning 
lasers)  These  parameters  can  normally  ba  traded  against  each 
other  in  any  specific  system  design.  A  comparison  of  several  existing 
projection  system  capabilities  is  shown  in  Figure  6.  It  is  noted  that 
the  scanning  laser  has  an  Image  Resolution  Factor  significantly 
above  the  rest  However,  if  a  third  factor  related  to  delivered 
brightness  is  considered,  the  light  valve  systems  rate  more 
favorably  relative  to  the  laser  systems.  For  the  purpose  of  this  paper, 
image  resolution  is  defined  conservatively  such  that  an  interfaced 
television  raster  display  consisting  of  1000  viewable  lines  subtending 
a  total  vertical  angle  of  36  degrees  to  the  eye  is  said  to  have  6  arc 
minutes  of  resolution  per  line  pair  (assuming  a  Kell  factor  of  .71). 


Figure  6.  Projector  Image  Resolution  Factor 


ADVANCED  DISPLAY  SYSTEM  CONCEPTS 


Four  feasible  display  system  concepts  can  be  postulated  that 
incorporate  one  or  more  of  the  technology  advancements  assessed 
above.  The  order  in  which  these  concepts  are  discussed  below  is 
unrelated  to  their  preference  as  potential  RSIS  simulator  designs. 
The  first  three  concepts  discussed  utilize  a  spherical  screen  the 
external  dimension  of  which  is  limited  to  20.5  ft  in  order  to  clear  the 
walls  of  the  VMS  facility  under  all  rotational  and  translational 
excursions.  The  performance  characteristics  given  are  for  selected 
display  projectors  only.  Any  of  the  projector  types  evaluated  in  Figure 
6  are  potentially  usable  although  system  efficiency  may  limit  use  of 
those  with  lower  light  output.  All  slewable  display  scenes  will  project 
60  degrees  downward. 


hi  Valve  and  Extension  Optics  Concept 


This  relatively  conventional  configuration  uses  three  light  valve 
projectors  combined  with  extension  optics.  The  concept  is  depicted 
in  Figure  7.  The  extended  "periscope"  design  of  the  optics  allows  the 
placement  of  lenses  near  the  center  of  a  spherical  screen  to 
minimize  distortion,  channel  matching  and  focus  problems.  A  head 
tracker  on  the  crew  member's  helmet  controls  the  motion  of  the 
projector  and  optics  in  the  pitch  axis.  Projector  images  are  edge 
matched  by  masking  inside  the  extension  optics.  Advantages  of  this 
concept  are  low  design  risk  and  high  scene  brightness. 
Disadvantages  are  lateral  FOV  limitations  and  resolution  marginaHty. 
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Figure  7.  Light  Valve  and  Extension  Optic 
Viaual  System  Concept 


Light  Valve  and  Fiber  Optics  Concept 

In  this  concept  flexible  coherent  fiber  optic  bundles  transmit 
images  to  an  optical  head  from  three  light  valve  projectors  fixed  to 
the  crew  station  platform.  The  fiber  optic  bundles  are  frequency 
multiplexed  to  minimize  the  effect  of  individual  fiber  breakage.  The 
optical  head  is  gimbaled  as  shown  in  Figure  8.  The  gimbal  is  slaved 
to  the  motion  of  the  crew  members  helmet  in  pitch  and  yaw  and 
rotates  about  the  exit  pupil  of  the  optics.  The  image  is  a  composite 
designed  with  high  resolution  in  the  central  area  of  the  display  by 
insetting  one  of  the  channels  of  6.5  arc  min/Lp  resolution  in  a  pair  of 
lower  resolution  fields.  Advantages  of  this  approach  include  reduced 
gimbal  drive  power  requirements  and  wider  total  field  of  view  than 
the  preceding  concept. 


Figure  8.  Light  Valva  and  Fiber  Optica 
Visual  System  Concept 


A  second  approach  (not  shown)  using  the  same  light  valve 
projectors  and  fiber  optics  eliminates  the  gimbal  and  adds  a  fourth 
channel  to  widen  the  instantaneous  field  of  view.  With  this 
arrangement  a  223-degree  horizontal  field  of  view  with  composite 
resolution  fields  is  feasible  at  a  brightness  reduction  to  about  2.3  fL. 
This  approach  increases  the  reliability  and  instantaneous  field  of 
view  but  decreases  the  resolution  on  the  sides  to  a  marginal  13  arc 
minutes.  There  is  also  an  undesirable  47- degree  long  horizontal 
"window''  joint  at  the  center  of  the  display  scene 

Scanning  Laser  Concept 

The  use  of  a  scanning  laser  allows  the  projection  of  a  bright 
collimated  beam  of  light  on  a  spherical  screen  with  a  vertical  raster 
scan  As  in  the  previous  concepts,  the  scanner  projector  is 


positioned  above  the  crew  member's  heads  as  shown  in  Figure  9. 
Because  of  the  large  depth  of  field  the  projector  is  not  constrained  to 
the  screen  center.  The  display  is  slewable  in  pitch  and  is  slaved  to 
helmet  position  in  pitch. 

Advantages  of  the  laser  concept  are  good  resolution  and  wide 
instantaneous  field  of  view.  The  large  continuous  scan  requires 
special  interface  considerations  with  computer  image  generation 
hardware. 


Figure  9.  Scanning  Laser  Visual  System  Concept 


Helmet  Mounted  Display  Concept 

In  this  concept  a  small  virtual  imaging  system  is  mounted  on  a 
crew  members  heimet.  Three  light  valve  projectors  relay  the  visual 
images  to  this  Helmet  Mounted  Display  (HMD)  via  flexible  coherent 
fiber  optic  bundles.  The  three  images  are  processed  optically  into 
two  scenes,  one  for  each  eye,  at  the  output  of  the  projectors.  The 
Sketch  in  Figure  10  depicts  a  concept  using  two  such  systems. 


Figure  10.  Helmet-Mounted  Display  Visual  System  Concept 


The  HMD  has  optical  combiner  lenses  which  permit  "viewing"  of  the 
Internal  cab  and  instruments  in  the  areas  of  view  where  the  CGI 
image  Is  blanked.  Prior  cockpit  mapping  provides  cab  interior  polar 
plot  information  to  blank  the  image.  An  artist's  concept  ol  this  HMD 
blanking  is  shown  in  Figure  1 1 .  A  head  tracking  system  provides  pilot 
head  position  information  to  the  CGI  visual  system. 
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The  advantages  of  the  HMD  approach  are  numerous.  The  concept 
offers  effectively  unlimited  total  field  of  view  with  a  minimum  of 
distortion.  Illumination  efficiency  is  adequate  to  allow  a  wide  range  of 
projector  possibilities.  Elimination  of  external  screen  or  other  optical 
elements  allows  a  large  space  and  weight  saving.  Disadvantages 
include  some  head  encumberance  and  some  incompatibility  with 
actual  aircraft  helmet  mounted  hardware.  However  of  all  the 
concepts  studied,  the  HMD  uses  the  newest  and  least  proven 
techniques  and  thus  involves  the  highest  risk. 
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Figure  11.  Helmet -Mounted  Visor  Olspfay  Blanking 


VISUAL  SYSTEM  CONCEPTS  EVALUATION 

Although  no  single  system  concept  (display  concept  and  image 
generator)  will  completely  meet  all  the  requirements  set  forth  by  the 
RSIS  program,  any  one  of  the  approaches  will  provide  a  substantial 
portion.  There  is  no  imminent  technology  break-through  that  will 
exceed  the  ACAVS  system  requirements.  Rapid  progress  is 
however,  being  made  in  the  areas  of  helmet  mounted  displays, 
scanned  laser  systems  and  computer  generated  imagery.  Some 
areas  such  as  HMD  will  need  to  be  addressed  in  further  detail  before 
a  design  decision  is  made  to  implement  them  in  a  system  design. 

Although  proposed  as  a  research  and  development  device, 
ACAVS  advanced  visual  system  concepts  employ  many  features 
that  would  seem  to  be  valuable  for  many  future  flight  trainers  and 
especially,  of  course,  rotorcratt  flight  trainers.  Their  wide  fields  of 
view  coupled  with  high  resolution  allow  a  wide  latitude  for  designers 
seeking  to  fulfill  difficult  training  requirements  associated  with  visual 
target  acquisition  and  tracking,  weapons  delivery  and  landing  as  well 
as  NOE  Dying  techniques. 

EXPECTED  BENEFITS 

The  Army/NASA,  1975  study14',  concluded  that  the  utilization  of  a 
helicopter  R&D  simulator  fell  into  two  categories: 

a.  Support  of  basic  technology  This  work  consists  of 
generalized  or  generic  studies  of  stability  and  control, 
handling  qualities,  controls  and  displays,  and  other 
aspects  of  the  man-machine  interface. 

b.  Support  of  the  development  of  new  aviation  systems  or 
improvements  to  fielded  systems:  These  efforts  start 
early  in  an  aircraft  acquisition  cycle  by  assisting  the  User 
and  the  Developer  in  performing  design  studies  and 
system  integration  evaluations  and  trade-offs. 

During  recent  years  a  coordinated  program  has  been  undertaken 
at  Ames  Research  Center  to  provide  a  data  base  for  helicopter 
handling  qualities  and  control  system  design  criteria.  As  indicated  in 
Figure  12.  advancements  in  rotor  systems  and  their  associated  flight 
controls  otter  the  most  direct  method  of  improving  flying  qualities  and 
reducing  pilot  workload  in  the  NOE  regimes  (Figure  1 )  where  Army 
helicopters  operate.  Chen  and  Talbot(13\  investigated  four  major 
rotor  system  design  parameters  to  assess  the  handling  qualities  for 
44  configurations  of  main-rotor  systems  which  cover  teetering, 
articulated  and  hingeless  families  of  rotor  systems  with  a  wide  range 
of  blade  inertia.  They  concluded  that  within  each  family  of  rotor 
systems,  satisfactory  handling  qualities  were  obtained  by 
appropriately  adjusting  the  rotor  parameters.  No  rotor  system  was 
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Figure  12.  Helicopter  Handling  Qualities  Research 

uniformly  superior  in  NOE  handling  qualities.  It  follows  that  additional 
experiments  will  be  required  to  optimize  the  handling  qualities  for 
specific  missions  conducted  in  the  NOE  environment. 

Aiken  and  Merrill1'4'  investigated  control  system  variations  for  an 
attack  helicopter  mission.  This  was  part  of  a  major  area  of  research 
aimed  at  reducing  pilot  workload  of  highly  maneuverable  helicopters 
that  are  intended  to  function  as  stable  platforms  for  target 
designation  or  weapon  defivery  at  night  or  under  adverse  weather 
conditions.  Two  candidate  techniques  are  under  investigation:  (1) 
modifications  to  the  control  system,  and  consequently  the  handling 
qualities,  as  a  result  of  different  flight  modes;  e.g.  -  cruise, 
approach  to  a  hover,  hover,  bob-up,  pop-up.  etc;  and  (2)  variations  in 
the  method  by  which  critical  information  is  displayed  to  the  pilot.  Both 
of  these  techniques  have  great  potential  for  reducing  pilot  workload 
and  additional  experiments  are  planned. 

The  uses  of  R&D  simulators  in  the  development  of  new  aviation 
systems  or  improvements  to  fielded  systems  follow  the  life  cycle  of 
system  development.  During  the  program  initiation  phase,  the 
simulator  can  be  used  to  evaluate  new  aviation  design  concepts  that 
have  been  developed  by  the  U.S.  Army  Training  and  Doctrine 
Command  (TRADOC),  to  meet  a  specific  threat.  This  evaluation  can 
help  answer  the  questions  and  support  the  rationale  leading  to  a 
Mission  Element  Needs  Statement  (MENS).  After  the  MENS  is 
approved,  the  R&D  simulator  can  be  used  in  the  demonstration  and 
validation  phase  for  evaluating  the  Hying  qualities  of  competing 
designs  as  well  as  the  ease  of  future  systems  integration  efforts 

Finally,  the  simulator  can  be  used  for  engineenng  development 
and  product  improvement.  As  the  design  evolves,  either  on  paper  or 
as  a  result  of  initial  flight  testing,  changes  to  correct  shortcomings  in 
either  flying  qualities  or  total  performance  can  be  evaluated  for  their 
effectiveness  as  well  as  their  undesirable  side-effects.  Project 
managers  can  also  use  the  simulator  to  evaluate  product 
improvement  proposals  prior  to  actual  hardware  fabrication  and 
expensive  flight  qualification. 

Flight  simulation  is  an  important  tool  in  helicopter  research  and 
development,  both  for  technology  base  development  and  for  aircraft 
development  programs.  The  RSIS  will  be  a  unique  capability  for  use 
by  both  government  and  industry  in  an  effort  to  maintain  our  current 
lead  In  helicopter  development  and  production  in  the  free  work). 
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ABSTRACT 

The  cost  and  effectiveness  of  computer-based  instruction  for  military  training  are  evalu¬ 
ated  on  the  basis  of  about  30  studies  conducted  since  1968.  Four  methods  of  instruction  are 
distinguished  and  compared:  Conventional  Instruction,  Individualized  Instruction,  Computer- 
Assisted  Instruction  ( CAI )  and  Computer-Managed  Instruction  (CMI).  Student  achievement  at 
school  is  about  the  same  with  all  methods  of  instruction.  CAI  and  CMI  save  about  30  percent 
(median)  of  the  time  required  by  students  to  complete  the  same  course  given  by  conventional 
instruction.  Individualized  instruction  (without  computer  support)  also  saves  student  time; 
the  addition  of  CAI  or  CMI  to  courses  taught  this  way  saves  little  additional  student  time. 
Student  attrition  appears  to  increase  with  CMI  compared  with  conventional  instruction,  but 
changes  in  student  quality  may  also  account  for  this  Increase;  no  such  data  are  available  on 
CAI.  Students  prefer  CAI  or  CMI  to  conventional  instruction;  attitudes  of  instructors,  con¬ 
sidered  in  only  a  few  studies,  are  unfavorable  to  CAI  and  CMI.  Direct  comparisons  of  the  cost 
and  effectiveness  of  different  methods  of  instruction  are  not  now  possible  because  of  Incomplete 
cost  data.  Most  so-called  cost  savings  attributed  to  CAI  and  CMI  are  based  on  estimates  of  pay 
and  allowances  of  students  for  the  time  saved  by  these  methods  of  instruction;  data  for  other 
costs,  such  as  for  CAI  or  CMI  equipment,  courseware  and  instructors  must  also  be  considered. 


INTRODUCTION 

The  training  of  military  personnel  is  a  large 
and  continuing  activity  for  the  military  Services. 
This  paper  evaluates  the  cost  and  effectiveness  of 
modern  training  methods  which  use  computers  to  in¬ 
struct  students  and  to  manage  students'  progress 
through  a  course.  (1) 

In  Fiscal  Year  1981,  about  428,000  enlisted 
personnel  will  receive  initial  skill  training, 
173,000  will  receive  skill  progression  training  and 
515,000  will  receive  functional  training  at  a  cost 
of  over  $2  billion.  (2)  About  70  percent  of  all 
recruits  complete  their  initial  enlistment;  less 
than  40  percent  reenlist  for  a  second  term;  thus 
about  75  percent  of  the  training  loads  and  costs 
noted  above  are  related  to  new  accessions  each 
year.  (2) 

After  graduating  from  technical  training 
schools,  personnel  are  assigned  to  operational 
commands  where  they  receive  additional  on-the-job 
training,  crew,  team  and  unit  training  and  prac¬ 
tice  in  field  exercises.  These  types  of  training 
have  large  costs  which  must  be  added  to  the  costs 
cited  above. 

Industry  is  well  aware  of  the  need  to  reduce 
the  costs  of  military  training  and  some  type  of 
computer-based  instruction  is  often  incorporated 
now  in  new  proposed  systems  such  as,  for  example, 
the  new  aircraft  training  system  (Navy  VTXTS)  and 
maintenance  simulators  (F-16).  This  paper  eval¬ 
uates  what  is  now  known  about  the  cost  and  effec¬ 
tiveness  of  various  types  of  computer-based 
Instructional  systems. 

METHODS  OF  INSTRUCTION 

Carpenter-Hufflnan  lists  20  different  methods 
of  Instruction  In  "Method  of  Designing  Instruc¬ 
tional  Alternatives"  (M0DIA)  (3);  these  Include 
lectures,  discussions,  tutoring.  Independent  study, 
drill  and  practice.  For  convenience,  we  have  orga¬ 
nized  these  methods  of  Instruction  Into  four 
groups;  more  than  one  method  of  instruction  may  be 
used  in  a  course. 


Conventional  Instruction.  Conventional  instruction 
refers  to  many  possible  combinations  of  lectures, 
discussions,  laboratory  exercises  and  tutorial 
sessions.  A  key  feature  of  conventional  instruction 
is  that  groups  of  students  proceed  through  a  course 
at  the  same  pace.  Differences  in  the  amount  of 
information  retained  by  students  are  reflected  in 
their  grades  at  the  end  of  the  course.  Conventional 
instruction  is  also  referred  to  as  lock-step  in¬ 
struction  and  group  scheduling.  It  is  used  in  75  to 
90  percent  of  all  military  courses,  although  a 
precise  estimate  is  not  available. 

Individualized  Instruction.  In  individualized  in¬ 
struction,  a  course  is  arranged  in  a  series  of 
lessons  through  which  each  student  proceeds  at  his 
own  pace.  Mastery  of  each  lesson  is  required  as  a 
condition  of  progress  and  there  Is  a  test  for  each 
lesson.  Differences  among  students  are  reflected 
in  the  amount  of  time  It  takes  them  to  complete  a 
course,  although  grades  may  also  be  given. 

Various  versions  of  individualized  Instruction 
may  differ  in  such  ways  as  the  amount  of  new  mate¬ 
rial  between  lessons,  the  order  in  which  lessons  are 
provided  to  the  student  (main  line,  branching)  and 
the  extent  to  which  the  student  is  completely  free 
to  proceed  at  his  own  pace.  All  methods  of  compu¬ 
ter-based  Instruction  rely  on  some  form  of  individu¬ 
alized  instruction;  the  term  "individualized 
instruction",  as  used  here,  is  meant  to  apply  only 
to  individualized  instruction  conducted  without 
computer  support.  The  terms  individualized  instruc¬ 
tion,  self-pacing,  and  programmed  Instruction  are 
often  used  synonymously. 

Computer-Assisted  Instruction  (CAI).  The  term  com¬ 
puter-based  instruction,  as  used  here.  Includes 
both  CAI  and  CMI.  In  computer-assisted  instruction 
(CAI),  the  student  interacts  in  real  time,  via  an 
interactive  terminal,  with  instructional  material 
that  is  stored  In  the  computer.  This  offers  great 
flexibility  'or  presenting  alternative  versions  of 
the  same  lessons  according  to  each  student's  parti¬ 
cular  ability  and  way  of  learning.  Most  CAI  systems 
diagnose  student  performance,  prescribe  lessons,  and 
maintain  student  records.  Examples  of  some  CAI 
systems  follow: 
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•  PLATO:  Programmed  Logic  for  Automatic 
Teaching  Operation.  The  current  version  of 
this  system,  PLATO  IV,  can  support  about 
950  terminals  linked  through  microwave  and 
land-line  communications  to  a  large  central 
computer  (COC  CYBER  74)  located  at  the  Uni¬ 
versity  of  Illinois. 

•  nCCTT:  Time-Shared  Interactive  Computer- 
controlled  Information-Television.  The 
basic  TICCIT  system  uses  one  or  two  mini¬ 
computers  to  support  up  to  128  terminals 
at  one  location. 

•  LTS:  Lincoln  Terminal  System.  The  latest 
version,  LTS-5,  uses  microfiche  to  store 
both  visual  images  and  an  audio  track. 

This  is  a  self-contained  or  "stand-alone" 
system. 

•  GETS:  General  Electric  Training  System. 

This  is  a  stand-alone  system  which  uses  a 
random  access  35-mm  slide  projector  for 
visual  displays  and  floppy  discs  for 
lessons  and  playback. 

Computer-Managed  Instruction  (CMI).  In  computer- 
managed  instruction  (CMl),  instruction  takes  place 
away  from  the  computer.  After  each  lesson,  the 
student  takes  a  test,  and  places  the  answer  sheet 
on  an  optical  scanner;  the  computer  scores  the  test 
and  interprets  the  results.  The  student  receives 
the  results  on  a  printout  which  tells  him  how  well 
he  performed,  what  lesson  to  take  next,  and  where 
to  find  it.  The  computer  also  manages  student 
records,  instructional  resources,  and  administra¬ 
tive  data.  Examples  of  some  CMI  systems  follow: 

•  M§.:  Advanced  Instructional  System.  This 
prototype  system  is  installed  at  the  Air 
Force  Technical  Training  Center,  Lowry 
Air  Force  Base,  Denver,  Colorado.  It  con¬ 
tains  50  student  terminals,  11  management 
terminals,  and  a  CDC  CYBER  73-16  computer 
which  can  support  up  to  3,000  students  a 
day  in  four  courses.  These  courses  were 
selected  to  represent  a  cross  section  of 
the  technical  training  courses  at  Lowry 
AFB  and  account  for  about  25  percent  of  the 
student  body.  The  management  terminals  are 
used  by  Instructors  for  developing  or 
revising  lessons  and  for  retrieving  data 
collected  by  the  system.  The  system  could 
be  expanded  to  provide  CAI  services  to 
students. 

•  Navy  CMI:  Computer  Managed  Instruction 
System.  This  system.  Installed  at  Naval  Air 
Technical  Training  Center,  Millington, 
Tennessee,  handles  about  6,000  students  in 
11  schools  at  5  training  centers.  It  is 
based  on  a  Honeywell  Series  60,  level  66 
computer. 

•  CTS:  Computerized  Training  System.  This 
system.  Installed  at  U.S.  Army  Signal  Center 
and  School,  Fort  Gordon,  Georgia,  can  pro¬ 
vide  CAI  and  CMI  services  for  128  terminals. 
It  is  based  on  six  mini -computers  (PDP- 
ll/35s).  Each  terminal  contains  a  visual 
display  unit  and  a  keyboard  which  can  pro¬ 
vide  both  interactive  instruction  and  course 
management  services. 


DISTINCTIONS  BETWEEN  MILITARY  AND 
'  NON-MILITARY  TRAINING 

Military  personnel  receive  pay  and  allowances 
while  they  are  bsing  trained.  Thus,  any  procedure 
which  can  reduce  the  length  of  time  required  for 
training,  without  significantly  affecting  the 
amount  and/or  quality  of  information  acquired,  can 
help  to  reduce  the  cost  of  training;  it  can  also 
help  to  increase  the  amount  of  time  spent  by  mili¬ 
tary  personnel  in  operational  assignments  during 
their  military  careers.  Military  training  courses 
are  designed  to  qualify  students  for  well-defined 
jobs  to  which  they  can  be  assigned  upon  successful 
completion  of  these  courses. 

This  situation  differs  from  that  found  in 
almost  all  types  of  public  and  private  schools 
where  students  are  not  paid  and  must  remain  at 
school  for  required  periods  of  time.  These  schools 
receive  no  direct  benefit  if  students  complete 
their  instruction  in  less  than  the  required  time. 
Courses  are  generally  not  designed  to  qualify 
students  for  particular  jobs  and,  obviously, 
schools  cannot  assign  students  to  jobs  when  they 
graduate. 

A  major  consequence  of  these  distinctions  is 
that  methods  of  instruction  that  are  cost-effective 
for  military  training  may  not  be  cost-effective  in 
other  applications.  Another  is  that  research  on 
computer-based  instruction  supported  by  the  mili¬ 
tary  Services  has  emphasized  the  possibility  of 
saving  student  time  while  maintaining  student 
achievement  constant.  Research  on  instruction  in 
non-military  settings  has  emphasized  student 
achievement  at  school  and  has  not  been  concerned 
with  the  amount  of  time  needed  by  students  to 
acquire  course  materials. 

THE  EFFECTIVENESS  OF  COMPUTER-BASED  INSTRUCTION 

The  military  Services  have  supported  research 
and  development  on  computer-based  instruction  since 
the  early  1960's  when  this  concept  first  appeared 
to  be  feasible.  The  use  of  CAI  and  CMI  in  military 
training  has  been  evaluated  in  about  30  studie? 
conducted  since  about  1968;  these  studies  provid< 

48  sets  of  data.  Most  of  these  studies  were  r  1 
research  nature  and  only  a  few  approximated 
operational  application.  The  nature  of  these 
studies  must  be  understood  in  order  to  interpret 
properly  the  data  they  provide. 

Military  training  is  intended  to  prepar  ■  pt ■ - 
sonnel  to  perform  various  jobs  in  operations  com¬ 
mands.  Thus,  the  effectiveness  of  any  method  of 
instruction  should  be  evaluated  by  measurina  how 
well  a  course  graduate  performs  certain  jobs  in 
the  field.  No  studies  provide  this  type  of  infor¬ 
mation.  Instead,  all  studies  of  different  methods 
of  instruction  compare  only  student  achievement  at 
school,  as  measured  by  tests  administered  there. 

This  limitation  applies  generally  to  most  research 
on  training  and  selection. 

Most  studies  were,  in  fact,  experiments. 

About  half  involved  50  or  fewer  students;  about 
half  the  courses  studied  lasted  one  week  or  less. 

In  a  few  studies,  the  courses  lasted  2  to  10  months 
and  involved  600  to  2000  students.  This  information 
is  summarized  in  Fig.  1.  A  wide  variety  of  subject 
matter  was  considered  In  these  courses  (see  Fig.  2); 
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NOTE:  Al  entries  in  table  refer  to  CAI  except  where  two  values  are  shown.  Then,  read  “CAI/CMI”. 
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FIGURE  1.  Distributions  of  Course  Length  and  Number  of  Students  in  48  EvaTuatipcis  of 

Military  CAI  and  CMI  Instruction 


lt-»7U 

FIGURE  2.  Courses  Used  in  Various 
Studies  of  CAI  and  CMI 


many  of  them  are  directly  related  to  technical 
training,  e.g.,  basic  electronics  and  electricity; 
some  involved  hands-on  maintenance  checkout  and 
repair.  Note,  however,  that  no  single  course  was 
given  both  by  CAI  and  CMI;  as  a  result,  this  means 
that  we  cannot  directly  compare  the  effectiveness  of 
CAI  and  CMI  for  giving  the  same  course. 

Student  Achievement  at  School.  Student  achievement 
at  school  with  CAI  was  about  the  same  as  with  con¬ 
ventional  instruction  in  24  cases,  superior  in  15, 
and  inferior  in  one  (Fig.  3).  The  differences  in 
achievement,  although  statistically  significant, 
were  not  judged  to  have  practical  significance. 
Student  achievement  at  school  with  CMI  was  about  the 
same  as  with  conventional  instruction  (8  cases). 

Amount  of  Student  Time  Saved.  Data  on  the  amount  of 
student  time  saved  by  CAI  and  CMI,  compared  to  con¬ 
ventional  instruction,  are  shown  in  Fig.  4.  The 
amounts  saved  range  from  -31  to  89  percent;  most 
values  are  positive;  the  median  value  is  about  30 
percent.  No  particular  significance  can  be  attrib¬ 
uted  to  differential  time  savings  between  CAI  and 
CMI  shown  in  these  data  because  direct  comparisons, 
using  the  same  courses,  were  not  made.  A  major 
uncontrolled  variable  in  these  studies  is  the  un¬ 
known  quality  of  the  Instructional  materials  used 
in  the  various  comparisons.  This  argues  against 
trying  to  make  quantitative  comparisons  about  the 
amount  of  student  time  saved  by  different  types  of 
CAI  or  CMI,  or  in  different  courses,  and  so  on. 

The  fact  that  CAI  and  CMI  save  student  training 
time  Is  consistent  with  the  we.l-known  fact  that 
there  are  wide  differences  in  student  ability 
(represented  by  the  normal  distribution  curve); 
there  are  also  differences  between  students  in  the 
amount  of  relevant  knowledge  held  by  them  at  the 
start  of  any  course.  In  conventional  instruction 
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FIGURE  3.  Student  Achievement  at  School  for  CAI  and  CMI,  Compared  to  Conventional 
Instruction,  in  Military  Training 
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FIGURE  4.  Amount  of  Student  Time  Saved  in  Courses  Given  by  CAI  and  CMI,  Compared 
to  Conventional  Instruction,  in  Military  Training 


with  a  fixed  amount  of  time,  these  differences  lead 
to  variations  in  the  amount  of  knowledge  held  by 
students  at  the  end  of  the  course,  i,e,,  as  shown 
by  a  distribution  of  final  grades.  In  individu¬ 
alized  instruction,  whether  computer-based  or  not, 
each  student  proceeds  at  his  own  pace  and  differ¬ 
ences  between  students  Influence  the  amount  of  time 
needed  to  acquire  the  course  materials.  Differences 


in  the  amount  of  information  acquired  is  not  a 
major  variable.  The  bulk  of  the  time  savings  in 
individualized  Instruction  is  produced  by  those 
students  for  whom  the  rate  of  progress  set  in  con¬ 
ventional  instruction  would  be  too  slow;  typically 
that  rate  might  be  one  that  permits  about  90  per¬ 
cent  of  the  students  to  complete  the  course  during 
the  fixed  period  of  time.  Figure  5  summarizes  the 
amounts  of  student  time  saved  shown  above. 
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FIGURE  5.  Amounts  of  Student  Training  Time 
Saved  by  CAI  and  CMI,  Compared  to 
Conventional  Instruction 


Course  Completion  Times  at  AIS  oyer  24  Months. 
Almost  all  of  the  data  shown  previously  represent 
time  savings  found  in  experiments  or  operational 
applications  over  short  time  periods  and  with  lim¬ 
ited  numbers  of  students.  Figure  6  shows  time 
saved  by  about  11,000  students  in  four  courses  on 
the  Air  Force  Advanced  Instructional  System  (AIS), 
Lowry  AFB  for  24  months  ending  September  1978.  It 
is  clear  that  the  initial  savings,  such  as  might  be 
reported  in  an  experiment,  are  maintained  over  time 
and,  despite  monthly  fluctuations,  tend  to  increase 
These  reductions  are  probably  due  to  periodic  revi¬ 
sions  in  the  courses  (indicated  on  the  figure)  and 
to  improved  control  over  the  new  method  of  instruc¬ 
tion;  fluctuations  are  probably  due,  at  least  in 
part,  to  variations  in  student  aptitude  and  to 
other  factors  that  are  presently  unknown. 


FIGURE  6.  Days  Required  to  Complete  Four  Courses  on  Air  Force 
Advanced  Instructional  System,  Lowry  AFB, 

October  1976  -  September  1978 
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FIGURE  7.  Amount  of  Student  Time  Saved,  Compared  to  Conventional  Instruction, 
by  Individualized  Instruction  and  by  CAI  or  CMI  on  the  Same  Courses 


Comparison  of  Time  Savings  Found  with  Indivi¬ 
dualized  Instruction  and  Computer-Based  Instruction. 
Student  training  time  in  courses  can  be  reduced 
without  the  use  of  computer-based  instruction, 
i.e.,  by  individualized  (or  self-paced)  Instruction 
without  computer  support  in  place  of  conventional 
instruction.  The  question  raised  here  is  "what 
does  computer  support  do  that  self-pacing  does  not 
do,  at  least  with  respect  to  the  amount  of  student 
time  saved?" 

Figure  7  shows  how  much  student  time  was 
required  to  complete  twelve  courses  that  were 
given,  at  various  times,  by  conventional,  individu¬ 
alized  and  CAI  or  CMI  Instruction.  Only  the  per¬ 
cent  time  saved,  compared  to  conventional 
instruction,  is  shown.  There  are  only  small  dif¬ 
ferences  between  the  amounts  of  student  time  saved 
by  these  three  methods  of  instruction;  the  data  are 
summarized  in  Fig.  8.  Individualized  instruction 
can  save  large  amounts  of  the  time  required  by 
students  in  conventional  instruction  (average 
savings  of  50  percent  or  more  in  these  samples). 

The  addition  of  CAI  to  five  individualized  courses 
produced  an  additional  average  time  saving  of  5 
percent;  the  addition  of  CMI  to  seven  courses 
produced  no  additional  average  time  saving.  No 
significance  can  be  placed  on  the  differential 
effect  of  adding  CAI  rather  than  CMI  to  the  base 
values  because  no  course  was  given  in  both  a  CAI 
and  CMI  mode.  The  addition  of  computer  support  may 
or  may  not  be  economical,  depending  on  its  impact, 
in  specific  applications,  on  the  numbers  of 
instructors  and  support  personnel,  administrative 
costs  and  similar  factors.  The  issue  of  cost  is 
not  addressed  by  these  data. 
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FIGURE  8.  Average  Amount  of  Student  Time  Saved 
by  Individualized  Instruction  and  CAI  or  CMI 
in  the  Same  Courses  Compared  to 
Conventional  Instruction 


Student  Attrition.  Since  the  method  of  instruction 
may  influence  the  number  of  students  who  can  suc¬ 
cessfully  complete  a  course,  the  rate  of  academic 
attrition  associated  with  alternative  methods  of 
instruction  should  be  a  matter  of  concern.  Mean¬ 
ingful  data  on  student  attrition  should  come  from  a 
steady-state  application  of  computer-based  instruc¬ 
tion  and  not  from  short-term  experiments.  This 
condition  was  met  marginally  by  the  Air  Force 
Advanced  Instructional  System  (A1S),  where  four 
courses  were  Increasingly  Implemented  In  a  CMI  mode 
over  the  period  of  1974-1978  and  by  the  Navy  Com¬ 
puter  Managed  Instruction  System,  where  data  are 
available  on  seven  courses  for  15  months  after  im¬ 
plementation  in  March  1977.  It  should  be  recog¬ 
nized  that  the  rate  of  attrition  observed  in  a 
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course  may  also  be  Influenced  from  time  to  time  by 
policy  decisions  on  standards  for  recruitment  and 
the  number  of  graduates  to  be  produced  by  various 
courses.  Such  Influences,  If  present,  are  not 
addressed  here. 

Figure  9  shows  that  academic  attrition  may 
have  Increased  in  the  four  courses  on  AIS  during 
the  period  under  observation.  Note,  however,  that 
academic  attrition  appeared  to  rise  In  all  (non- 
AIS)  courses  at  Lowry  AF8  over  the  same  period; 
thus.  It  Is  not  necessarily  true  that  the  Increased 
attrition  in  the  AIS  courses  can  be  attributed 
primarily  to  the  introduction. of  CMI. 

Figure  10  shows  academic  attrition  for  seven 
courses  before  and  after  Implementation  on  the  Navy 
Computer  Managed  Instruction  System.  The  average 
rate  of  attrition  in  these  courses  was  3.2  percent 
before  and  4.6  percent  after  Implementation  In  a 
CMI  mode;  it  increased  in  six  courses  and  decreased 
in  one.  Data  on  comparable  courses  not  on  CMI 
during  the  same  period  were  not  provided. 


So  far,  only  two  CMI  systems,  the  Air  Force 
AIS  and  Navy  CMI,  have  received  some  extended, 
though  still  limited,  use  In  military  training. 
Academic  attrition  may  have  increased  In  courses 
taught  by  CMI  compared  to  attrition  with  conven¬ 
tional  Instruction.  Since  these  comparisons  do  not 
take  Into  account  possible  changes  in  the  qualifi¬ 
cations  of  students  during  the  periods  under 
observation,  the  available  data  suggest  but  do  not 
prove  that  CMI  may  Increase  academic  attrition  over 
that  found  with  conventional  instruction. 

Attitudes  of  Students  and  Instructors.  Attitudes 
of  students  and  instructors  to  CAI  or  CMI,  in  com¬ 
parison  with  conventional  instruction,  are  noted 
here  only  as  qualitative  aspects  of  these  methods 
of  Instruction;  they  are  not  direct  measures  of 
effectiveness.  Data  on  student  attitudes  are  found 
in  39  of  the  48  reports  of  military  training  sum¬ 
marized  in  this  paper.  As  shown  in  Figure  11, 
students  overwhelmingly  prefer  CAI  or  CMI  to  con¬ 
ventional  instruction,  or  at  least  say  so  when 
asked;  students  are  favorable  to  CAI  in  29  of  40 


FIGURE  9.  Academic  Elimination  Rate  in  Four  Courses  Befors  and  After 
Implementation  on  AIS  at  Lowry  AFR 
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FIGURE  10.  Academic  Attrition  in  Seven  Courses  Before  and  After  Implementation 
on  Navy  Computer  Managed  Instruction  System 
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FIGURE  11.  Attitudes  of  Students  and  Instructors  Comparing  CAI  or  CMI 
to  Conventional  Instruction  in  Military  Training  (a) 


comparisons;  they  are  unfavorable  in  one  case  and 
find  no  difference  in  another;  no  data  are  provided 
in  9  cases;  they  are  favorable  to  CMI  In  all  cases 
(N=8).  Instructor  attitudes  are  reported  only  in 
9  of  these  48  comparisons;  instructors  are  un¬ 
favorable  to  CAI  or  CMI  In  8  of  these  cases  and 
favorable  to  CAI  only  in  one. 

Instructors  of  courses  taught  by  CAI  or  CMI 
have  not  received  much  attention  by  researchers. 


The  training  of  Instructors  Is  still  oriented 
largely  towards  conventional  Instruction  and 
Instructors  receive  little  guidance  on  how  to 
properly  conduct  CAI  or  CMI  courses. 

THE  COST  OF  COMPUTER-BASED  INSTRUCTION 

The  cost  of  computer-based  instruction  is  not 
a  popular  subject  for  examination;  only  eight  of 
the  30  studies  cited  above  provide  any  cost  data; 


132 


i 


RESOURCE  (TYPE  OR  FUNCTION) 

Program  Development 
Program  Design 

Instructional  Materials:bConventional  Instruction 
huHviduaMzed  Instruction 
Programming 
First  Unit  Production11 
Computer-Based  Instruction 


_ Coding 

Program  Delvery 
Instruction:  Instructors 

Instructional  Support  Personnel 
Equipment  and  Services:e  Laboratory  (Incl.  simulators) 
Media  Devices 
Computer  Systems 
Communications 
Materials  (incl.  Consumables )' 

Faculties# 

Program  Management  and  Administration 
Student  Personnel:  Pay  and  Allowances 


Others  (PCS,  TOY,  art) 


NOTE:  ShaM  ctls  an  not  appScaMa.  Blank  colt  indicate  that  retevaat  cost  data  am  not  aiauw 
■tectados  TICCtT,  BM  1500,  LTS-3,  GETS,  and  an  axpnknontal  shipboard  system. 

Hnctedas  revlstan. 
cMaster  copy. 

*A3  direct  ptrsonnol  oat  kidudtd  in  sthor  catefoitos. 

*tedudos  al  hardware  rotated  costs:  Initial  (Including  kisttSltian  and  clwckout),  mod  location,  and  roplacoment;  oporatlon  and  maintenance; 
lease  and  user  loos;  computer  system  software;  orb 
tnctados  copies  of  instructional  materials  (hooks,  courseware  copies,  art). 

(Structures,  natures,  and  furnishings. 

h Laboratory  equipment  and  media  devices  are  appacaUo  to  al  methods  of  Instruction  (except  whore  simulated  in  CM  systems),  and  there 
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FIGURE  12.  Number  of  Sources  of  Data  on  Costs  of  Instruction 


only  five  consider  the  cost-effectiveness  of  com¬ 
puter-based  Instruction. 

Data  on  the  Costs  of  Instruction.  Figure  12 
Identifies  the  various  resources  needed  to  conduct 
Instruction,  broken  down  according  to  type  or 
function.  All  methods  of  Instruction  require 
resources  (for  example,  for  program  design, 
Instructors,  materials  and  facilities);  CAI  and 
CMI,  In  particular,  require  resources  for  computer 
systems  and  software.  Relevant  cost  data  could  be 
found  for  some  but  not  all  of  these  types  of 
resources.  Figure  12  shows  the  number  of  sources 
of  data  we  were  able  to  find  on  the  costs  of  each 
of  these  resources  for  various  methods  of  Instruc¬ 
tion;  blank  spots  indicate  that  relevant  cost  data 
could  not  be  found.  The  data  that  were  found  were 
meager  at  best.  These  data  are  summarized  In  many 
tables  In  the  original  report.  (1)  The  following 
points  may  be  noted:  (a)  There  are  few  data  on  the 


costs  of  conventional  instruction  and  individu¬ 
alized  instruction,  (b)  More  cost  data  could  be 
provided  by  converting  student  time  savings  Into 
cost  savings,  on  the  basis  of  student  pay  and 
allowances,  (c)  There  is  no  convenient  way  to 
summarize  the  available  cost  data  in  a  single 
table. 

The  Cost  of  Computer  Systems  Hardware.  There  may 
be  some  Interest  In  discussing  the  costs  of  central 
processors  and  terminal  units,  for  selected 
Instructional  systems.  The  costs  shown  in  Fig.  13 
are  limited  to  hardware  and  do  not  include  the 
costs  of  preparing  Instructional  materials,  pro¬ 
gramming,  instructors  and  the  like;  note  that  the 
cost  data  are  for  1978  and  some  earlier  dates. 

System  hardware  costs  can  be  expressed  in 
three  ways:  (a)  system  procurement  cost,  (b)  cost 
per  terminal  connected,  and  (c)  cost  per  student-hour 
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FIGURE  13.  Costs  of  Computer  Systems  Hardware 


(over  some  chosen  amortization  period).  In  terms 
of  computer  system  hardware,  procurement  costs  can 
range  from  about  $35,000  (the  stand-alone  GETS)  to 
over  $10  million  (a  1,000-terminal  PLATO  IV  sys¬ 
tem),  a  factor  of  close  to  300  times.  On  a  per- 
terminal  basis,  there  appears  to  be  an  inverse 
relationship  between  system  size  and  cost.  As  an 
example,  for  the  TICCIT  system,  the  per-terminal 
system  cost  is  $27,000  in  a  32-terminal  configu¬ 
ration  and  $10,000  in  a  128-terminal  configuration 
This  indicates  a  substantial  economy  of  scale  for 
larger  systems. 

A  more  meaningful  relationship  for  comparing 
computer-btsed  instruction  with  other  methods  of 
instruction  is  the  cost  per  student-hour.  Assume 
that  terminals  are  used  2000  hours  per  year  and 
that  the  system  hardware  lasts  5  years;  then,  the 
cost  of  hardware  per  student  hour  for  CAI  systems 
ranges  from  about  $1.00  (for  the  128- terminal 
TICCIT  system)  to  $3.50  (GETS);  PLATO  IV  costs 
about  $1.50.  The  lower  cost  per  student-hour 
associated  with  large  systems  Implies  a  large 
initial  commitment  of  funds  (if  central  hardware 
is  purchased)  and  a  large  commitment  to  CAI,  with 
the  other  costs  and  risks  it  entails.  Assuming 
that  each  CMI  terminal  would  accommodate  50 
students,  student-hour  costs  for  CMI  would  appear 
to  be  less  than  $0.10. 

Note  that  the  $3.50  associated  with  the  GETS 
Is  based  on  information  that  Is  several  years  old. 
Systems  of  comparable  capability,  incorporating 
recent  technological  advances  in  microprocessors 
and  data  storage  devices,  can  be  anticipated  to 
cost  considerably  less. 


THE  COST-EFFECTIVENESS  OF  CAI  AND  CMI 

There  have  been  only  a  few  attempts  to  esti¬ 
mate  the  cost-effectiveness  of  CAI  and  CMI  and 
these  are  based  on  incomplete  analyses  of  the  costs 
of  instruction.  Figure  14  summarizes  the  results 
of  these  studies.  All  of  these  studies  are  based 
on  the  premise  that  the  amount  of  student  training 
time  saved  by  a  method  of  Instruction  indicates 
major  cost  savings;  the  amounts  of  cost  savings  are 
estimated  by  computing  the  pay  and  allowances  of 
students  for  the  amounts  of  student  time  saved  in 
training.  The  resultant  amounts  should  more  prop¬ 
erly  be  called  "cost  avoidance  savings".  This 
procedure  was  applied  to  student  time  savings  in 
studies  of  PLATO  IV,  Navy  CMI,  and  AIS  and,  in  one 
case,  to  revised  course  materials  in  a  course  given 
by  conventional  Instruction.  Four  of  these  studies 
Include  some  other  costs,  such  as  for  preparing 
course  materials,  purchase  or  use  of  computers,  and 
the  number  of  Instructors  required  by  each  method 
of  instruction. 

The  dollar  amounts  of  such  "savings”  could  be 
large,  depending,  of  course,  on  the  number  of 
students  assumed  for  these  estimates,  e.g.,  about 
$10  million  a  year  for  about  50,000  students 
Instructed  in  FY  1977  by  the  Navy  CMI  system  and 
about  $3  million  a  year  for  about  5,500  students 
instructed  In  FY  1978  by  the  Air  Force  AIS  system. 
According  to  two  cost-effectiveness  evaluations 
that  have  been  reported,  the  PLATO  IV  system  was 
judged  to  be  not  as  cost-effective  as  individu¬ 
alized  instruction.  These  conclusions  are  based  on 
Incomplete  cost  data  in  two  small-scale  tests  (535 
students  in  four  courses  at  U.S.  Army  Ordnance 
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FIGURE  14.  Summary  of  Studies  Reporting  Cost  Savings  and/or 
Cost-Effectiveness  of  Various  Methods  of  Instruction 


Center  and  School,  Aberdeen  Proving  Ground,  Mary¬ 
land,  1975;  1261  students  in  four  courses  at 
Chanute  AFB,  Illinois,  1977).  The  Air  Force  AIS 
was  found  to  be  cost-effective,  compared  to 
instructor-supported,  self-paced  Instruction  In  one 
course  (Inventory  Management)  but  not  in  three 
others;  the  computer  costs  which  made  the  latter 
courses  not  cost-effective  were  judged  to  be  small 
in  comparison  to  other  school  costs  (AIS  Service 
Test,  1978).  Since  all  of  these  results  are  based 
on  Incomplete  cost  data,  the  findings  should  not  be 
generalized  or  taken  as  conclusive. 

Other  benefits,  beyond  those  of  saving  student 
training  time,  are  often  said  to  occur  with  CAI  or 
CMI,  because  of  the  many  services  that  can  be 
provided  by  a  computer.  The  following  list  illus¬ 
trates  a  variety  of  claims  that  have  been  made: 

1.  More  precise  data  for  improving  and  up¬ 
dating  course  materials 

2.  Improved  control  over  equipment,  facili¬ 
ties  and  materials  for  instruction 

3.  Improved  allocation  of  resources  among 
students 

4.  Improved  ability  to  accommodate  fluctu¬ 
ations  in  student  loads 


5.  Reduced  instructorrstudent  ratios  (in¬ 
cluding  the  ability  to  use  less-qualified 
Instructors) 

6.  Reduced  need  for  support  by  non- 
instructional  personnel 

7.  Reduced  time  of  students  on  base  waiting 
for  courses  to  start 

8.  Reduced  time  of  students  on  base  waiting 
for  travel  orders  after  completing  courses 

9.  Improved  integration  of  personnel  records 
of  students  at  school 

10.  Improved  utilization  of  Instructors. 

Few  of  these  potential  benefits  have  been  in¬ 
cluded  In  any  cost-effectiveness  evaluation  known 
to  us.  The  AIS  Service  Test  (1978)  estimated  the 
numbers  and  costs  of  Instructors  required  for  indi¬ 
vidualized  and  computer-managed  instruction  by  ob¬ 
serving  what  Instructors  were  doing  In  each  case. 
Records  kept  on  the  AIS  show  that  the  amounts  of 
time  spent  by  students  at  Lowry  AFB  waiting  to 
enter  a  course  and  waiting  for  an  assignment  after 
completing  a  course  have  been  reduced  for  those  in¬ 
structed  by  AIS.  Records  kept  by  the  Navy  CMI  sys¬ 
tem  show  that,  because  of  course  time  reductions. 
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the  average  on  board  count  of  students  In  school 
has  been  reduced  for  those  instructed  by  that  sys¬ 
tem.  However,  the  cost  savings,  if  any,  implied 
by  these  reductions  were  not  included  in  any  of  the 
cost-effectiveness  analyses. 

CONCLUSIONS 

1.  Measures  of  Effectiveness.  The  effectiveness 
of  computer-assisted  and  computer-managed  instruc¬ 
tion  for  military  training  has  been  measured  only 
by  student  achievement  at  school  and  not  by  per¬ 
formance  on  the  job.  Correlations  between  perform¬ 
ance  at  school  and  on  the  job  have  not  been  estab¬ 
lished  for  any  method  of  instruction. 


in  attrition  may  also  be  due  to  decreases  in 
student  quality;  the  influence  of  this  factor  has 
not  been  carefully  examined.  No  data  have  been 
reported  on  student  attrition  with  computer- 
assisted  instruction. 

6.  Student  and  Instructor  Attitudes.  Attitudes  of 
students  toward  computer-assisted  and  computer- 
managed  instruction  appear  to  be  favorable.  Atti¬ 
tudes  of  Instructors  are  reported  as  unfavorable, 
but  this  finding  is  based  on  very  limited  data. 
Little  attention  has  been  given  to  the  role  of 
instructors  in  computer-based  instruction  and  to 
how  they  should  be  prepared  for  this  method  of 
instruction. 


2.  Student  Achievement  at  School.  Student 
achievement  in  courses  at  military  training  schools 
with  computer-assisted  instruction  is  the  same  as 
or  greater  than  that  with  conventional  instruction; 
the  amount  of  additional  achievement  is  small  and 
has  little  practical  importance.  Student  achieve¬ 
ment  in  courses  with  computer-managed  instruction 
is  about  the  same  as  that  with  conventional  in¬ 
struction.  Both  of  these  results  are  due  to 
keeping  students  in  CAI  and  CMI  courses  until  they 
achieve  standards  set  previously  for  conventional 
instruction. 


3.  Student  Time  Savings.  Computer-assisted  and 
computer-managed  instruction  in  military  training 
save  about  30  percent  of  the  time  (median  value) 
needed  by  students  to  complete  the  same  courses 
given  by  conventional  instruction.  The  amounts  of 
student  time  saved  by  computer-based  instruction 
vary  widely,  but  little  attention  has  been  given 
to  the  factors  that  could  account  for  the  wide 
variation.  Most  of  the  results  on  computer- 
assisted  instruction  come  from  experiments  of 
limited  duration,  with  limited  amounts  of  course 
materials,  and  with  relatively  few  students. 

Where  computer-managed  instruction  has  been  used 
for  extended  periods  (up  to  4  years),  the  initial 
time  savings  have  been  maintained  or  increased. 


4.  Individualized  and  Computer-Based  Instruction. 
Individualized  instruction  (self-paced  instruction 
without  computer-support)  saves  student  time; 
little  or  no  additional  student  time  is  saved  when 


the  same  courses  are  given  by  computer-assisted  or 
computer-managed  instruction. 


5.  Student  Attrition.  Computer-managed  instruc¬ 
tion  may  increase  the  rate  of  student  attrition  for 
academic  reasons,  compared  to  that  with  conven¬ 
tional  instruction.  However,  the  observed  Increases 


7.  Cost  Data.  Only  limited  and  incomplete  data 
are  available  on  the  costs  of  computer-assisted 
and  computer-managed  instruction  in  military 
training.  Data  that  are  collected  routinely  on 
the  costs  of  operational  training  programs  are  too 
highly  aggregated,  particularly  with  respect  to 
training  support  functions,  for  use  in  analytical 
comparisons  of  computer-based  instruction  with 
conventional  instruction. 

8.  Cost-effectiveness.  Estimates  based  on  the 
amounts  of  student  time  saved  suggest  that  the  Navy 
Computer  Managed  Instruction  System  avoided  costs 
of  about  $10  million  in  FY  1977  and  that  the  Air 
Force  Advanced  Instructional  System  avoided  costs 
of  about  $3  million  in  FY  1978.  These  estimates 
are  Incomplete  because  they  do  not  consider  all  of 
the  costs  of  providing  computer-managed  instruction 
at  these  installations  or  compare  these  costs  with 
the  costs  of  alternative  methods  of  instruction  for 
the  same  courses. 
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ESTABLISHING  THE  TRAINING  COST  FOR  A  COMPLEX  WEAPONS  SYSTEM: 
AN  EXAMPLE  USING  THE  P-3  FRS 
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San  Diego,  CA  92131 


and 


Walter  F.  Thode 

Navy  Personnel  Research  and  Development  Center 
San  Diego,  CA  92152 


This  paper  describes  an  approach  of  establishing  the  training  cost 
for  a  complex  weapons  system.  The  Navy  P-3  Training  Program  in  VP-30 
and  VP-31  was  used  as  a  model  in  establishing  the  cost  parameters.  In 
this  program  over  5,691  training  objectives  were  developed  stretching 
over  seven  crew  positions  and  a  multitude  of  training  tracks.  This 
paper  will  show  how  costs  were  developed  for  each  crew  position 
describing  the  costs  for  learning  center,  for  weapon  system  trainer, 
for  cockpit  procedure  trainer,  for  position  trainer  and  for  the 
aircraft  itself. 


INTRODUCTION 


Problem  and  Overview 

This  paper  describes,  using  the 
example  of  the  Navy's  ASW  Patrol  Fleet 
Replacement  Squadron  (FRS),  Patrol 
Squadron  31  (VP-31),  a  model  for  esti¬ 
mating  the  training  costs  for  a  complex 
weapons  system.  It  is  not  a  definitive 
list  of  all  costs  nor  does  it  attempt  to 
account  for  all  costs  in  the  Patrol  FRS. 
It  does,  however,  account  for  the  major 
cost  contributors  and  give  a  framework 
for  gathering  and  estimating  them. 

The  manager  for  a  complex  weapons 
system  training  organization  currently 
makes  major  decisions  affecting  syllabus 
changes  without,  in  many  cases,  real¬ 
izing  the  impact  of  these  decisions  on 
the  total  training  system  cost.  A 
training  squadron  often  has  cognizance 
of  but  a  limited  part  of  the  total 
syllabus  costs,  usually  aircraft  direct 
operating  costs.  The  costs  for  the 
instructor  salaries,  training  devices, 
Instructional  Systems  Development  (ISD) 
efforts,  and  student  time  are  not 
visible,  and  thus  are  often  discounted. 

Consideration  by  the  training 
manager  should  be  given  to  the  costs  in 
making  changes  to  the  syllabus  as  well 
as  their  Instructional  implications. 

The  Defense  Science  Board  placed  consi¬ 
derable  emphasis  on  the  need  to  make 
cost  benefit  analysis  when  making 
training  management  decisions  (1). 


Background 

Previous  studies  have  been  found 
that  bring  a  methodology  to  bear  on  the 
problem  in  part,  but  none  have  applica¬ 
tion  for  the  operational  user.  Doughty, 
Stern,  and  Thompson  (2)  have  developed  a 
guideline  for  cost  analysis  in  a  typical 
U.S.  Navy  "A"  School  but  their  study  did 
not  adequately  reflect  the  ISD  acquisi¬ 
tion  costs,  or  the  costs  relevant  to 
more  exotic  hands-on  media  such  as 
flight  simulators  and  aircraft  flight 
hours.  They  also  do  not  cover  multiple 
training  tracks  and  the  interaction  of 
media  versus  training  tracks  resulting 
from  multicrewed  systems.  Orlansky  and 
String  (3,  4)  give  a  rather  complete 
generalized  model  for  estimating  train¬ 
ing  costs.  However,  they  do  not  apply 
their  model  in  a  complete  manner  to  a 
specific  Navy  training  application. 
Allbee  and  Semple  (5)  have  developed  an 
extensive  cost  model  using  various  U.S. 
Air  Force  examples  which  do  not  apply  to 
Navy  fleet  training  due  to  the  differing 
data  source  bases  between  Air  Force  and 
Navy  data  files.  The  Allbee  and  Semple 
cost  model  does  render  valuable  insights 
for  the  development  of  cost  models. 
However,  the  model  is  too  detailed  for 
use  by  fleet  training  squadrons,  in  that 
it  takes  into  account  some  variables 
that  are  not  useful  at  the  FRS.  An 
example  of  this  is  the  factoring  into 
the  model  the  costs  of  such  items  as  the 
Air  Staff  and  Higher  Headquarters. 


Existing  studies  approaching  the 
data  for  fleet  level  use  are  either 
outdated  or  incomplete.  Browning,  Ryan, 
Scott,  and  Smode  (6)  were  not  tasked  to 
look  at  any  media  except  the  2F87F 
simulator,  and  did  not  make  comparisons 
in  other  than  the  pilot  training  track. 
The  cost  data  in  the  study  were  also 
based  on  July  1976  dollars.  Braby, 
Henry,  and  Morris  (7)  indicate  some 
guidelines  for  costing  media  alterna¬ 
tives  that  are  helpful  in  isolating 
costs  of  conventional  media.  Goclowski , 
King,  Ronco,  and  Arkren  (9)  have 
described  a  model  identifying  system 
ownership  costs  (SOC).  Their  SOC  model 
is  a  good  reference  in  attempting  to 
establish  a  generality  for  identifying 
cost  drivers  in  a  complex  training 
system . 

The  present  study  is  not  a  new 
approach  to  the  data  in  the  aggregate 
but  represents  a  refinement  of  existing 
models  for  the  user  in  the  fleet 
readiness  squadron. 


DEVELOPMENT  OF  THE  MODEL  FOR 
ESTIMATING  P-3  TRAINING  COSTS 


This  section  describes  a  model  for 
estimating  the  training  costs  for  the  VP 
Fleet  Replacement  Squadrons  ( FRS ) ,  VP-30 
and  31.  It  includes  certain  simplfying 
assumptions  that  will  be  discussed 
below.  However,  it  is  important  to  note 
that  the  major  cost  drivers  have  been 
identified  and  are  shown  as  variables. 
This  model  follows  the  construct  of 
Orlansky  and  String  (3,  4)  in  that  it 
does  not  display  sensitivity  to  the 
timing  of  costs  or  to  budget  implica¬ 
tions  of  program  alternatives.  It 
serves  as  a  general  structure  for 
estimating  training  costs. 


Student  Load 

The  determination  of  student  quotas 
for  this  model  was  reported  by  Thode 
(10).  The  quotas  are  displayed  for  a 
one-year  period  by  position  for  one  FRS, 
and  for  both  FRSs.  Also  shown  are  data 


for  15  years.  15  years  is  required  for 
later  calculations  for  estimating  the 
revision  costs  of  the  ISP  material  and 
the  life-cycle  costs  of  certain  media. 


TABLE  1 


NOMINAL  STUDENT  LOAD 


1  YEAR 

15  YEA 

POSITION 

One  FRS 

Both  FRS 

Pilot 

180 

360 

5,400 

NFO 

120 

240 

3,600 

FE 

140 

280 

4,200 

SS  1/2 

120 

240 

3,600 

SS  3 

60 

120 

1,800 

COMM 

40 

80 

1,200 

ORD 

120 

240 

3,600 

Total  all 

years 

23,400 

A  simplifying  assumption  has  been  made 

as  it  pertains  to  student  load.  Student 
attrition  has  not  been  accounted  for. 

It  is  estimated  that  attrition  is  of 
minimal  significance  for  the  VP  cost 
model.  If,  however,  attrition  is  of 
program  significance  it  can  be  accommo¬ 
dated  by  adding  to  the  student  load  an 
additional  student  quota  that  represents 
the  impact  of  each  loss  on  the  total 
load.  For  a  more  complete  discussion  of 
attrition  see  Orlansky  and  String  (3). 

Description  of  Demand  Created  bv  Current 
Syllabus - ~ - 1 - 

The  Master  Course  Syllabus  (MCS)  (8) 
describes  each  hour  of  instruction  for 
all  media  and  crew  positions.  Crew 
positions  are  described  by  type  of 
aircraft  (P-3B  or  P-3C)  as  well  as  by 
job  title.  The  MCS  is  summarized  in 
Table  2.  The  number  of  instructional 
objectives  for  each  crew  position  is 
taken  from  CO  VP-31  LTR  (11).  The  media 
shown  include  learning  center  activi¬ 
ties,  the  numerical  designations  of 
training  devices  and  simulators,  and  the 
P-3  itself. 


TABLE  2 


SUMMARY  OF 

MASTER  COURSE 

SYLLABUS 

CREW  POSITION 

#  OBJECTIVES 

SYLLABUS 

HOURS 

BY  MEDIA 

Total  LC 

2C45/69 

2F87F 

14B44  2F87T 

P-3 

Pilot 

1st  Tour 

2nd  Tour3 

1000  248 

22 

28 

3  32 

95  ' B-20 ) 

239 

9 

30 

32 

50  B-15) 

»B  =  B 

Stage  Hours 

139 


i 


i 


M 


TABLE  2  (Continued) 


CREW  POSITION  t  OBJECTIVES  SYLLABUS  HOURS  BY  MEDIA 


LC 

2C95/69 

2F87F  Static/A/C 

P-3 

FEb 

413 

265 

8 

36  1 1 

104  ( B-40 ) 

LC 

P-3  LAB 

2F69  2F87T 

P-3 

NFO  Ba 

843 

252 

21 

69 

70 

Ca 

869 

386 

27 

74 

54 

LC 

19B99 

2F69  2F87T 

A/C  LAB  P-3 

SS  1 /2  B 

575 

236 

57 

32 

20 

36 

C 

605 

229 

54 

32 

35 

30 

A/C 

LC  14B40  2F69 

2F87T  1 5  Z 1 

15E16 

Ui tE  P-3 

SS  3  B 

298 

98 

32 

9 

1 1 

26  70 

C 

366 

154 

36 

32 

25  70 

LC 

Static/A/C  A/C  LAB 

P-_ 

3 

ORD  B 

199 

99 

33 

18 

24 

C 

209 

90 

31 

15 

24 

COMM  B 

345 

130 

17 

51 

30 

TOTAL  -- 

Hours  of 

Developed 

Instruction  by  Media 

Number  of  Learning  Objectives 

Learning  Center  Hours  of 
Instruction 

Syllabus  Hours  Assigned  to 


5,691 

2,321 

2C45/69 

79 

2F87F 

94 

2F87T 

205 

2F69 (T ) 

133 

1  4B44 

114 

P-? 

657 

P-3  (Lab) 

228 

P-?  (Static) 

92 

1  4340 

36 

15Z  1 

9 

15E16 

1  1 

Notes  : 

a  s  Tactical  Team  Training  An  Additional  54  hours 
b  2  NAMTRADET  an  Additional  160  hours 

c  =  Pilot  1st  and  2nd  tour  objections  are  not  differentiated 


Table  3  shows  the  instructor-to- 
student  ratio  required  by  various 
generic  hands-on  media.  This  ratio  will 
be  used  later  in  the  cost  estimating 
section.  The  cost  for  the  learning 
center  is  described  along  with  the 
assumptions  in  the  cost  estimating 
section . 

The  instructor/student  ratio  is  only 
part  of  the  resource  factor  for  a  P-3 
sortie.  A  similar  concern  is  the  number 
of  crew  positions  that  benefit  from  any 
flight  training  hour.  An  analysis  of 


the  MCS  suggests  the  following 
allocation  of  flight  hours  per  position. 

Proportion 

Position  of  Flight  Hours 

MCS  Phase  B 

Pilot  .5 

FE  .5 

MCS  Phase  CAD 

~ m. -  .  i« 
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TABLE  3 


INSTRUCTOR/STUDENT  RATIO  FOR  HANDS-ON  MEDIA 


Position 

OFT/PT 

WST 

P-3 

1.  Pilot 

.5 

.5 

1 

2.  NFO 

1 

1 

1 

3.  FE 

.5 

.5 

.5 

4.  SS  1/2 

.25 

.5 

.5 

5.  SS  3 

.5 

1 

.5 

6.  COMM 

— 

-- 

.5 

7.  ORP 

— 

.3 

.5 

REF:  Welch  (12) 

(For  example:  The  ratio  of  a  syllabus  flight  hour  for  a 
pilot  student  to  an  instructor  flight  hour  is  1  to  1). 


For  Phase  B  flight  hours  during 
familiarization  the  MCS  shows  that  one 
pilot  and  one  FE  are  trained.  There¬ 
fore,  each  should  share  in  the  demand 
for  that  flight  hour,  and  the  proportion 
for  each  is  one-half  (.5).  In  C  and  D 
stages,  however,  all  crew  members  are 
receiving  training  of  some  type,  and  it 
is  assumed  that  the  flight  hour  costs 
are  spread  across  all  positions.  This 
data  will  then  be  used  in  the  establish¬ 
ment  of  costs  for  flight  hours. 


Estimating  Training  Costs 

The  example  shown  below  is  a  method 
of  establishing  training  costs.  Certain 
simplifying  assumptions  have  been  made, 
and  they  will  be  discussed.  The  cost 
"driver"  will  be  discussed  in  the  five 
sections.  Section  1  provides  the  costs 
of  instructional  systems  development. 
Section  2  shows  the  learning  center 
costs.  The  training  device  costs  are 
given  in  Section  3.  Section  4  shows  the 
P-3  aircraft  costs,  and  Section  5 
provides  the  instructor  costs. 

The  costs  shown  below  are  an 
approximation  of  the  "level  of  effort" 
for  each  cost  category.  They  show  the 
approximate  cost  "drivers"  in  the  VP 
FRS.  It  was  deemed  impractical  to 
attempt  to  ascertain  the  small  cost 
differences  among  some  P-3  model  types, 
e.g.,  one  P-3C  was  considered,  not  all 
the  variations  of  P-3C,  Update  I,  Update 
II  and  so  forth.  Also,  all  pilot  costs 
assumed  the  P-3C,  not  P-jB  or  update 
syllabi . 


Other  simplifying  assumptions  are  as 
follows: 

1.  Due  to  the  lack  of  data,  costs 
were  not  identified  for  VP-30 
and  31  instructor  training.  The 
high  turnover  of  pilot  instruc¬ 
tors  might  make  this  a  cost 
category  that  should  be 
developed  in  the  future. 

2.  As  was  explained  in  an  earlier 
section,  student  attrition  was 
assumed  to  be  zero.  If  there  is 
in  fact  a  significant  student 
attrition  in  some  tracks,  the 
time  where  attrition  occurs  and 
the  percentage  of  course 
completion  needs  to  factored 
into  the  student  costs 

3.  Administrative  overhead,  flights 
aborted  due  to  equipment 
malfunctions,  out  of  maintenance 
checK  flights,  and  instructor 

NA TOPS/ instrument  check  flights 
have  not  been  included  as 
student  flight  hours. 

4.  Except  as  noted,  costs  have  not 
been  established  for  some 
student  categories  such  as  2nd 
tour  NFO,  2nd  tour  FE,  NARF  test 
pilots,  or  other  rare 
categories  . 

b.  The  total  overhead  of  indirect 
administration  has  been  elimi¬ 
nated  for  simplicity  although  it 
could  be  added  if  desired.  The 
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overall  proportional  costs 
between  media,  however,  would 
probably  remain  unchanged.  Some 
of  the  administrative  overhead 
factors  that  were  not  addressed 
are  Squadron,  Wing,  COMNAVAIRPAC , 
and  FASOTRAGRUPAC ,  administration 
and  headquarters  costs.  Allbee 
and  Semple  (5)  gives  an  example 
of  USAF  training  cost  and  does 
include  costs  of  some  of  the 
higher  headquarters. 

6.  P-3  aircraft  costs  were  devel¬ 
oped  as  shown  from  the  NALCOMIS 
(Commander  Naval  Air  Systems 
Command)  report  (13),  but  depre¬ 
ciation  was  not  added  to  the 
calculation.  Depreciation  is 
deemed  to  be  not  relevant  in 
consideration  of  the  cost  of  the 
P-3.  Obsolescence  due  to  its 
mission  capability  probably 
occurs  prior  to  its  max  airframe 
life.  However,  if  aircraft 
depreciation  is  desired, 

Orlansky  (3)  shows  programmed 
flying  hours  to  be  “129  per  year, 
aircraft  acquisition  $8,280,000 
and  service  life  15  years.  This 
yields  a  straight  line  deprecia¬ 
tion  of  $552,000  per  year  or 
$1287  per  aircraft  hour. 

7.  Costs  for  military  construction 
have  not  been  included  in  the 
model  except  for  the  modifica¬ 
tion  to  the  learning  center. 

The  Moffett  learning  center 
costs  were  included  for  they 
were  a  major  cost  in  the 
establishment  of  the  revised  P-3 
curriculum.  It  was  also  assumed 
that  the  learning  center  costs 
at  NAS  Moffett  Field  were 
representative  of  the  coses  at 
NAS  Jacksonville,  FLA. 

8.  Costs  were  assumed  to  be  in 
constant  dollars.  It  is  obvious 
that  they  are  rapidly  changing 
in  the  current  environment. 
However,  for  the  purposes  of 
this  model  relative  costs  across 
media  are  important,  not  abso¬ 
lute  costs.  For  the  same  reason 
costs  were  not  discounted  for 
future  years  in  accordance  with 
DODI  7041. 3- (14) 

9.  Allowance  has  not  been  made  for 
any  major  changes  in  the  future 
to  the  P-3  aircraft.  If 
perchance,  a  P-3D  or  P-3C  Update 
IV  is  developed  the  ISD  update 
costs  for  new  equipment  or  a  new 
aircraft  (e.g.,  P— 4A )  would 
require  a  new  cost  estimation. 

The  addition  of  a  new  aircraft, 
new  equipment,  or  major  software 
changes  to  the  existing  aircraft 
requires  a  reevaluation  of  the 
costs.  The  current  aircraft  mix 
except  where  noted  for  Table  6 


represents  the  base  line  for 
this  report. 

10.  All  data  shown  in  the  report 
except  ISD  costs  (Table  8)  are 
for  NAS  Moffett  Field,  Cali¬ 
fornia,  only  and  costs  for  NAS 
Jacksonville,  Florida,  are 
probably  about  the  same.  It  is, 
therefore,  assumed  that  there 
are  not  significant  differences 
between  training  sites. 

Instructional  System  Development 

Costs 

The  ISD  cost  will  be  discussed  in 
two  general  areas:  first,  the  initial 
ISD  cost;  and  second,  the  costs  required 
to  maintain  curriculum  currency.  These 
two  cost  areas  will  then  be  displayed  in 
a  table  showing  the  ISD  costs  per 
student . 

Initial  Costs 

The  initial  contractor  cost  for  the 
P-3  activities  was  determined  from  the 
contract.  The  VP  ISD  team  personnel 
costs,  table  9,  were  developed  based  on 
Welch  (12)  for  the  staffing  and  man- 
years,  and  by  using  Koehler  (15,  16)  for 
billet  costs.  The  research  psychologist/ 
educational  specialist  man-years  and 
billet  costs  are  from  Thode  (10). 

The  category  of  other  media  costs 
shown  in  table  5  accounts  for  audio¬ 
visual  services  performed  by  the  U.S. 
Navy,  and  for  reproduction  costs  of  the 
A/V  productions  (10).  CO  VP-31  LTR  (11) 
is  the  reference  for  all  other  data  in 
table  5. 

Maintenance  Costs 

A  training  program  for  a  fleet 
aircraft  is  not  an  unchanging,  static 
syllabus  of  instruction.  Walker  (17) 
evaluated  an  ISD  activity  for  training 
crew  members  to  fly  a  U.S.  Navy  fleet 
aircraft  (S-3A)  with  a  similar  mission 
to  the  P-3.  The  results  of  this  study 
showed  that  over  a  two-year  period  22 
percent  of  the  media  required  change 
either  due  to  content,  learning  strategy 
or  media  related  issues.  Table  6 
assumes  that  a  significant  effort  is 
involved  in  maintenance  of  the 
curriculum,  an  effort  which  decreases 
over  time.  For  it  is  assumed  that 
during  years  1-5  the  current  ISD  staff 
shown  on  table  6  will  be  required  to 
revise/update  the  instruction.  The 
staffing  plan  for  the  initial  years 
after  introduction  of  the  ISD  syllabus 
was  based,  in  part,  on  recommendations 
developed  by  Walker  (17).  (NPRDC 
developed  the  staffing  recommendations 
( 1 8 ) . )  For  years  6-10  the  requirement 
for  the  maintenance  of  P-3B  instruction 
has  been  deleted  from  the  revision 
staff,  and  for  years  11-15  only  minor 
revisions  of  instruction  for  crew 
members  of  the  P-3C  are  assumed. 
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TABLE  4 


VP  ISD  TEAM  PERSONNEL  COSTS 


RANK /RATE 


DESIGNATION/ 

RATING 


0-4 

1320 

8 

$  270,072 

0-3 

1310 

16 

1,205,552 

0-3 

1320  (P-3B) 

17 

496,366 

0-3 

1320  (P-3C) 

16 

467,168 

E-7 

AW 

4 

107,060 

E-6 

AW  SS/3B 

6 

138,828 

E-6 

AW  SS/3C 

6 

138,828 

E-6 

AW  SS  1/2 

17 

393,346 

E-6 

AO 

8 

223,024 

E-7 

AD 

3 

95,370 

E-6 

AD 

10 

280,340 

E-5 

AT 

4 

94,488 

E-5 

DM 

4 

63,276 

E-3 

DM 

6 

71,100 

Civilian 

GS  11/12 

13 

700,000 

$4,544,818 

NPRDC 

GS-12  Research  Psychologists  $50,000  10  Man-years  $500,000 

NOTE:  Personnel  based  on  personal  communication  with  Thode  (10) 

and  LCDR  Welch  (12). 


TABLE  5 

ISD  DEVELOPMENT  COSTS  SUMMARY 


Courseware  Contract 

VP  ISD  Team  Personnel  Costs 

Other  Media  Costs 


TOTAL  #  of  Training  Objectives  in 
VP  Curriculum 

TOTAL  #  of  Hours  of  Instruction  in 
VP  Curriculum  All  Tracks 


Development  Cost 
Instructional  Hours 


$7.986.236 


$2,947,626 

4,544,818 

503,792 

$7,996,236 

5,691 

5,463 

Cost  Per  Hour  for 
Development 

$1,464 
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TABLE  6 

ISO  UPDATE  COSTS 

VP  31/FASO  ISD  UPDATE  PERSONNEL  YEAR  1-5 


Billet 

#_  Rank/Rate  Designator  Title  VP-31  FASO  Cost 

Administrative-Support  Staff 


1 

LCDR 

1320 

ISD  Team  Director 

X 

$  33,759 

1 

LCDR 

1310 

Asst  Team  Director 

X 

69,559 

1 

LT 

1310 

Syllabus  Director 

X 

75,347 

1 

LT 

1320 

Syllabus  Director 

X 

29,198 

SUB  TOTAL 

$207,863 

SUBJECT  MATTER 

EXPERT 

1 

LT 

1310 

B  Pilot 

X 

$  75,347 

1 

LT 

1310 

C  Pilot 

X 

75,347 

2 

LT 

1320 

ft  NFO 

X 

58,596 

1 

LT 

1320 

B  NFO 

X 

29,198 

2 

LT 

1 120 

C  NFO 

X 

58,596 

1 

LT 

1320 

C  NFO 

X 

29,198 

3 

AW-1 

SS  1/2 

X 

69,414 

2 

AW-  1 

SS  1/2 

X 

46,276 

1 

AWC 

SS  3 

X 

26,765 

23,138 

1 

AW-1 

SS  3 

X 

1 

AW-1 

SS  3 

X 

23,138 

1 

AOC 

C  ORD 

X 

34,342 

1/2 

AO-1 

B  ORD 

X 

13,939 

1 

ADCS 

FE 

X 

33,933 

1 

AD-1 

FE 

X 

28,034 

1 

AR-1 

B  Comm 

X 

30,689 

SHE  SUB  TOTAL 

$655,950 

Civilian  Staff 


1  GS-11  Educational  Specialist 

1  Civilian  Editor 

$24,200 
14  ,600 

$3MD6 

Total  Military  costs 

Cost  Adjusted  by  .58  Factor  (see  text) 
Adjusted  Cost 

Civilian  Staff  Cost 

Yearly  Cost  (years  1-5) 

$863,813 

-362,801 

5tn|M5 

38,800 

$BTOT? 

II  Rank/Rate  Designator  Title 

Administrative-Support  Staff 

VP-31  FASO 

Billet 

Cost 

1  LCTR  1320  ISD  Team  Director  x  $  33,759 
1  LCDR  1310  Asst  Team  Director  x  69,559 
1  6T  1320  Syllabus  Director  x  29,198 


ADMINISTRATIVE  SUB  TOTAL  $132,516 
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TABLE  6  (Continued) 


VP 

-31/FAS0 

ISD  UPDATE 

PERSONNEL 

YEAR  6-10 

Subject  Matter 

Experts 

1 

LT 

1310 

C  Pilot 

X 

$  75,347 

2 

LT 

1320 

C  NFC 

X 

58,596 

1 

LT 

1320 

C  NFO 

X 

29, 198 

3 

AW-  1 

S3  1/2 

X 

69,414 

2 

AW- 1 

S3  1/2 

X 

46,276 

1 

AWC 

SS  3 

X 

26,765 

1 

AW-1 

SS  3 

X 

23,138 

1 

AW- 1 

22  3 

X 

23,138 

1 

A0C 

C  ORD 

X 

34,342 

1 

ADC  3 

FE 

X 

33,933 

1 

AD-1 

FE 

X 

28,034 

SME  SUB 

TOTAL 

$448, 181 

Civilian  Staff 


GS-11  Educational  Specialist 

$24,200 

Civilian  Editor 

14  ,600 

$59,866 

Total  Military  Costs 

$580,697 

Cost  Adjustedgby  .58  Factor  (see  text) 

-243,893 

Adjusted  Coste 

336,804 

Civilian  Staff  Cost 

38,800 

Yearly  Cost  (years  6-10) 

664 

Billet 


#  Rank/Rate 

Designator 

Title  VP 

'-31 

FASO 

Cost 

Administrative 

Support  Staff 

1  LCDR 

1320 

ISD  Team  Leader 

X 

$  33,759 

1  LT 

1310 

Syllabus  Director 

X 

75,347 

ADMINISTRATIVE  SUB 

TOTAL 

$T0Y,  106 

Subject  Matter 

Experts 

1  LT 

1310 

C  Pilot 

X 

$  75,347 

1  LT 

1320 

C  NFO 

X 

29,198 

1  LT 

1320 

C  NFO 

X 

29,198 

1  AW-1 

SS  1/2 

X 

23,138 

1  AW-1 

SS  1/2 

X 

23,138 

1  AWC 

SS  3 

X 

26,765 

1  AW-1 

SS  3 

X 

23,138 

1  AOC 

C  ORD 

X 

34,342 

1  AD-1 

FE 

X 

28,034 

SME  SUB  TOTAL 

Civilian  Staff 

1  GS-1 1  Educational  Specialist/ 

Editor 

Total  Military  Costs 

Cost  Adjusted  by  .58  Factor  (see  text) 

Adjusted  Cost 

Civilian  Staff 

Yearly  Cost  (years  11-15) 
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$211,200 


401 , 404 
168,590 

$2l7firnr 

38,800 
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TABLE  6  (Continued) 

VP-31 /FASO  ISD  UPDATE  PERSONNEL  YEAR  11-15 


TOTAL  ISD  UPDATE  COSTS 


Year  Per  Year 


Total 


I- 5  $539,812 

6-10  375,604 

II- 15  276,614 


$2,699,060 

1,818,020 

1,383,070 


$5”,  956,  i5A 


Table  6  thus  portrays  the  cost  for 
the  VP-31/FAS0TRAGRUPAC  ISD  Update  based 
on  the  current  staffing,  and  on  future 
projections.  The  staffing  is  shown  in 
three  categories,  administrative 
support,  subject  matter  experts,  and 
civilian.  A  pair  of  columns  show  if  the 
staff  position  is  at  VP-31  or 
FASOTRAGRUPAC .  The  billet  costs  are 
from  Koehler  (15,  16). 

The  military  billet  costs  as  they 
affect  this  model  have  been  reduced  by 
.58.  NPRDC  (18)  has  calculated  that 
each  military  member  of  the  ISD  staff 
will  be  devoting  150  days  per  year  to 
revision/update  duties.  Since  there  are 
260  working  days  in  a  year  this,  then, 
works  out  to  approximately  a  ratio  of 
.58  to  one  man-year.  Military  duties, 
leave,  TAD,  flying,  and  other  duties 
account  for  the  remaining  days. 

ISD  Cost  Summary 

The  total  ISD  costs,  then,  can  be 
developed  by  adding  the  total  costs  of 


TABLE 


TOTAL  ISD 


Development  $ 

Update 


TOTAL  $1 


the  development  and  the  costs  of  the 
maintenance  requirements.  The  ISD  cost 
per  instructional  hour  or  per  learning 
objective  is  found  by  dividing  the 
number  of  hours  or  objectives  into  the 
total  ISD  cost  as  shown  in  table  7. 

The  cost  per  student  for  ISD  is  then 
found  by: 

1.  Determining  the  percentage  a 
specific  track  is  of  the  total 
training  (in  this  case  by 
percentage  of  training  objec¬ 
tives  represented  by  each 
training  track).  The  number  of 
objectives  per  track  is  found  in 
table  2. 

2.  Determining  the  total  number  of 
students  in  each  track  over  the 
15  years  of  estimated  life  from 
table  1. 

3.  Dividing  cost  per  track  by  the 
total  number  of  students  in  the 
track. 


COSTS 


,966,236 
,560,  150 


,556,386 


ISD  Cost 

Instructional  Hour 


Total  ISD 

Cost  Total  ISD  Cost  .  Total  ISD 

Per  Hour  Objective  =  Cost  Per  Obj. 


556,386 


$2,481 


jl ^.556, 386  _  $2,382 
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These  data  showing  cost  per  student 
are  shown  for  each  track  in  table  8. 

For  simplicity  no  differentiation  was 
made  between  P-3B,  P-3C,  Update  I,  II, 
etc.  The  ISD  costs  for  each  aircraft 
type  overlap  a  great  deal  and  their 
relative  cost  differences  would  not 
greatly  change  the  overall  relationship 
among  media. 

Learning  Center  Costs 

FASOTRAGRUPAC  maintains  a  learning 
center  at  VP-31  and  both  VP-31  and 
FASOTRAGRUPAC  provide  instructors  to 
support  the  instruction.  The  costs  for 
the  learning  center  (LC)  have  been 
allocated  in  the  following  manner. 
Although  this  calculation  is  shown  for 
one  site  the  costs  for  both  sites  is 
assumed  to  be  the  same. 


Mllcon  and  Equipment.  The  learning 
center  has  198  carrels.  For  this  model 
it  is  assumed  that  the  LC  and  its 
associated  equipment  will  have  a  10  year 
useful  life.  The  initial  mllcon  and 
equipment  costs  were  about  $600,000. 

The  investment  costs  then  of  the  LC  are 


$60,000  per  year.  For  further  discus¬ 
sion  about  mileon  costs  see  Allbee  and 
Semple  (5). 

LC  Support  Costs.  The  LC  support 
costrrsTTrrr  are  reported  below  and 
were  supplied  by  FASOTRAGRUPAC  (19). 
FASOTRAGRUPAC  reports  military  costs  by 
using  NAVC0MPN0TE  7041,  the  composite 
standard  military  rate  table.  This 
table  only  accounts  for  pay  and 
allowances,  not  the  overhead  which  is 
represented  in  the  Koehler  (15,  16). 

The  latter  appear  to  be  a  more  accurate 
reflection  of  the  actual  costs.  An 
example  is  that  they  include  the  cost 
for  pilot  training  for  the  billet  cost 
of  a  pilot,  NFO  training  for  NFO,  etc. 

If  NAVCOMPNOTE  7041  is  used  for  a 
Lieutenant  it  shows  a  yearly  cost  to  be 
$24,611;  Koehler  (15)  shows  a  pilot  to 
cost  $75,347  and  a  NFO  to  cost  $29,198 
per  year.  Billet  costs  for  civilians 
rate  are  under  development  and  should  be 
used  in  future  models  (20).  The 
FASOTRAGRUPAC  (19)  figure  for  overhead 
for  civilians  is  too  low  at  10$.  A 
sample  check  with  Koehler  (20)  shows 
that  to  be  an  underestimate  of  about 
10-20  percent. 


TABLE  8 

ISD  COST  PER  CREW  POSITION 


$  of  Training 

$  of 

Total 

Cost  Per 

Position 

Objec.  .ves 

Cost 

Students 

Student  ($) 

Pilot 

26 

$  3,524,660 

5400 

653 

NFO 

23 

3,117,969 

3600 

866 

FE 

1 1 

1,491,202 

4200 

355 

SS  1/2 

16 

2, 169,022 

3600 

603 

SS  3 

10 

1,355,639 

1800 

753 

COMM 

9 

1,220,075 

1200 

1017 

ORD 

5 

677,219 

3600 

188 

TOTAL 

,  COST 

$13,556,386 

The  cost  categories  are:  graphics 
personnel,  duplication  personnel  and 
supplies,  and  LC  staff.  The  graphics 
and  duplication  personnel  are  required 
to  update  and  reproduce  the  AV  and 
printed  instructional  materials.  The 
FASOTRAGRUPAC  budget  for  learning  center 


applicable  supplies  is  added  at  this 
point  to  LC  costs.  The  LC  staff 
consists  of  those  civilian  and  military 
personnel  who  issue  materials  and 
maintain  the  learning  center.  Table  9 
shows  the  cost  of  the  LC  not  including 
the  instructors. 
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TABLE  9 


LEARNING 
Graphics  Personnel  at 

CENTER  SUPPORT 

FASO  Moffett 

COSTS  PER  YEAR 

GRADE/RATE 

NUMBER 

PER  YEAR 

OVERHEAD 

TOTAL 

GS-9 

1 

18739 

10* 

$  20,612 

GS-7 

4 

15317 

10* 

67,394 

GS-5 

2 

12368 

10* 

27,209 

GS-4 

1 

11054 

10* 

12, 159 

TOTAL 

$127,376 

Duplication 

Personnel 

at  FASO  Moffett 

LI  2 

3 

15567 

INC 

$  46,701 

SN  (LI) 

2 

11848 

INC 

23,676 

TOTAL 

$  70,397 

LC  Supplies 

$  60,000 

Learning  Center  Staff 

Civilian 

GS-5 

1 

12368 

10* 

$  13,604 

GS-3 

1 

9846 

10* 

10,830 

GS-2 

2 

8902 

10* 

19,584 

GS-4 

1 

1  1054 

10* 

12,159 

Military 

YN  2 

1 

15347 

INC 

15,347 

AZ  3 

1 

14971 

INC 

14,971 

AA  (AZ) 

3 

13878 

INC 

41,634 

TOTAL 

$128,131 

Total  Noninstructor  Learning  Center  Costs  Per  Year 

MILCON  and  Equipment  Per  Year 

$  60,000 

Graphics  Personnel 

127,376 

Duplication 

Personnel 

70,397 

Supplies 

60,000 

LC  Staff 

128, 131 

Total  LC  noninstructor  cost 
total  Number  of  Students 


$326029 

- 


$418 


TOTAL  $326,024 

Total  noninstructor  cost  per  student 
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Instructor  Costs 

The  learning  center  instructor 
personnel  costs  are  shown  in  table  10. 
Since  both  FASO  and  VP-31  at  differing 
times  provide  instructors  to  the  LC 
depending  on  student  progress  in  the 
MCS,  it  was  assumed  that  a  full  time 
instructor  man-year  equivalent  was 
required  in  the  LC  for  each  track.  The 


assumed  instructor  rank/rate  is  also 
shown.  Koehler  (15,  16)  again  are  the 
billet  cost  references. 

Learning  Center  Cost  Summary 

Table  11  shows  the  total  cost  of  the 
LC  for  each  crew  position  accounting  for 
both  noninstructor  (NI)  and  instructor 
costs  per  student. 


TABLE  10 

LEARNING  CENTER  INSTRUCTOR  PERSONNEL  COSTS 


Crew 

Position 

Instructor 

Rank/Rate  Cost  Per  Year 

Students 

Per  Year 

Cost  Per 
Student 

Pilot 

LT 

$75 , 347 

180 

$419 

NFO 

LT 

29,189 

120 

243 

F£ 

E-6 

28,034 

140 

200 

SS  1/2 

E-6 

23, 138 

120 

193 

SS  3 

E-5 

18,662 

60 

311 

Comm 

E-5 

23,622 

120 

197 

ORD 

E-5 

18,602 

40 

465 

Total 

780 

TABLE  11 

TOTAL  LEARNING  CENTER  COSTS 
AVERAGED  OVER  ALL  STUDENTS, 

PER  STUDENT 
ALL  TRACKS 

Crew  Position 

Noninstructor  Instructor  Cost 

Total 

Pilot 

$418 

$419 

$837 

NFO 

418 

243 

661 

FE 

418 

200 

618 

SS  1/2 

418 

193 

611 

SS  3 

418 

311 

729 

Rad  io 

418 

197 

615 

ORD 

418 

465 

883 

NOTE: 

Costs  per  hour  can  be  found  by  dividing  the  total  LC  cost  by  the 
number  of  hours  in  the  MCS,  i.e.,  for  pilot  LC  hours  total  248 ; 
cost  $837,  therefore  cost  is  $3.38  per  hour. 
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Training  Device  Costs 

Training  device  costs  have  two  major 
components,  acquisition  costs  and 
operating  costs.  AXlbee  and  Semple  (5) 
discuss  in  their  model  an  elaborate 
means  for  estimating  these  costs  in  an 
Air  Force  setting.  Orlansky  and  String 
(3)  take  a  somewhat  simpler  view  of 
arriving  at  the  same  costs.  Browning, 
et.  al.,  (6)  estimates  the  cost  for  only 
the  training  devices  required  for  pilot 
training.  This  model,  then,  is  an 
elaboration  of  the  Orlansky  and  String 
model . 

Training  Device  Acquisition  Costs . 
Table  12  shows  the  training  device 
acquisition  costs.  The  unit  prices  were 


obtained  from  the  Chief  of  Naval 
Education  and  Training  (CNET),  directory 
of  Naval  Training  Devices  (21).  The 
unit  price  was  amortized  over  15  years 
to  calculate  the  price  per  year. 

Fifteen  years  of  service  life  for  a 
device  is  assumed.  It  should  be  noted, 
however,  that  the  unit  price  reflected 
in  the  CNET  report  is  probably  low  in 
that  it  does  not  reflect  costs  from 
trainer  engineering  changes  and  costs 
from  changes  or  modifications  to  the 
P-3.  The  operating  hours  were  reported 
by  FASOTRAGRUPAC  to  the  NALCOMIS 
reporting  system.  The  yearly  operating 
hours  authorized  were  then  divided  into 
the  acquisition  cost  per  year  in  order 
to  calculate  the  acquisition  cost  per 
hour . 


TABLE  12 

TRAINING  DEVICE  ACQUISITION  COSTS 
AT  NAS  MOFFETT  FIELD,  CA 


Trainer 

Unit  Price 

Price  Per  Year 
If  Amortized 

15  Years 

FASO 

Operating 

Hours 

Acquisition 
Cost  Per 

Hour  Per  Year 

2F87  (F) • 

$4,278,760 

$285,250 

8756 

$65. 16 

2F87  (T) • 

4,646,882 

309,792 

8023 

77.22 

14BH0 

2,081,970 

138,798 

1512 

91.80 

1  4B44 

2,764,113 

184,274 

2526 

72.95 

2C45A 

1,232,740 

82, 183 

1423 

57.75 

2F69C 

4,270,000 

284,667 

3139 

90.69 

•Two  devices  at  Moffett. 

NOTE:  Certain  obsolete  devices  were  deleted  from  this  table  due 

to  insufficient  data  being  available,  e.g.,  15  Z  1. 


TABLE  13 

FASOTRAGRUPAC  REPORTED  OPERATING  COSTS 


FY79 

Trainers  Costs 

2F87  (F)  $326,083 

2F87  (T)  277,608 

1 14BM0  89,607 

1 MB4«  98,323 

57,078 
321,311 


Authorized 
Operating 
Hours _ 

8,756 


8,023 


1,512 

2,526 

1,423 

3,139 


Operating 
Costs  Per 
Hour _ 

$  37 

35 

60 

39 

40 


2C45A 

2F69E 


150 


102 


Operating  Costa 

Table  13  depicts  the  FASOTRAGRUPAC 
FY  1979  reported  operating  costs  per 
device  (19). 

Training  Device  Cost  Summary 

Table  19  shows  a  summary  of  the 
annual  operating  and  acquisition  costs 
per  device  per  year  from  tables  12  and 
13. 


Although  the  annual  cost  per  device 
represents  a  significant  cost  in  this 
model,  the  cost  per  student  hour  must  be 
considered.  An  assumption  is  made  that 
there  is  no  slack  variable,  that  is, 
when  a  student  seat  is  available  for 


assignment  it  will  be  filled.  If  this 
assumption  is  used,  then  the  cost  per 
seat  in  the  model  can  be  found  by  adding 
the  acquisition  cost  per  hour  (table  12) 
and  operating  cost  per  hour  (table  13) 
to  obtain  this  total  cost  per  hour  per 
seat.  Then,  the  number  of  seats  per 
device  are  divided  into  the  cost  per 
hour  to  obtain  cost  per  student  hour  per 
seat  (table  15).  If  a  seat  is  available 
for  scheduling  but  not  used,  it  creates 
a  negative  cost  to  the  model,  adding  to 
the  real  cost.  The  actual  student  usage 
might  only  be  80  percent  of  the 
authorized  operating  hours.  If  this  is 
so,  it  raises  the  true  cost  of  a  student 
hour.  In  order  to  make  this  model  as 
workable  as  possible,  the  simplifying 
assumption  is  made  to  ignore  this 
factor . 


TABLE  14 


TOTAL  TRAINING  DEVICE  OPERATING  AND 
ACQUISITION  COST  PER  YEAR  PER  DEVICE 


t  Trainer 

Yearly 

Amorti zed 

Acquisition 

Cost 

FY  79 

Operation  A 
Maintenance 
Cost 

Yearly 

Costs 

(2  )2F87  (F) 

*570,500 

$326,083 

$896,583 

(2  )2F87  (T) 

619,584 

277,608 

897, 192 

(1  )  1 4B40 

138,798 

89,607 

228,405 

(1  )  1 4B44 

184,274 

98,323 

282,597 

(1  )2C45A 

82, 183 

57,018 

139,261 

(1  )2F69E 

284,667 

321,311 

605,978 

TABLE  15 

TRAINING  DEVICE 
COST  PER  STUDENT  SEAT  HOUR 


Trainer 

Acquisition 
Cost  Per 

Hour 

Operating 
Cost  Per 
Hour 

Total  Cost 
Per  Hour 

Total  Number 
Student  Seats 

Cost  Per 

Student 

Hour 

2F87F 

$65 

*  37 

$102 

3 

$34 

2F87T 

77 

35 

1  12 

5 

22 

14B40 

92 

60 

152 

3 

51 

14B44 

73 

39 

1 12 

6 

19 

2C45A 

58 

40 

98 

3 

33 

2F69E 

81 

102 

200 

8 

25 

Instructor  cost  for  each 
instructional  medium,  table  16,  was 
obtained  by  using  the  billet  costs  from 
Koehler  05)  for  enlisted  billet  costs 
and  Koehler  (16)  for  the  officer  billet 
costs.  The  rate/rank  of  the  instructors 
are  shown  in  the  assumptions.  The 
proportional  cost  of  each  instructor  to 
each  medium  was  taken  from  table  3,  The 
following  assumptions  were  used  in  the 
development  of  table  16. 

1.  Pilot  instructor  is  an  0-3 

2.  NFO  instructor  is  an  0-3 

3.  FE  instructor  is  an  E-6  AD 

4.  SS  1/2  instructor  is  an  E-6  AW 

5.  SS  3  instructor  is  an  E-5  AW 

6.  COMM  instructor  is  an  E-5  AT 

7.  ORD  instructor  is  an  E-5  AO 

8.  2,000  hours  per  instructor  man 
year 


Aircraft  Flight/Lab  Costs 

The  cost  per  hour  for  the  P-3C 
aircraft  was  determined  from  the 
NALCOMIS  report  (13).  First,  the  total 
annual  flying  hours  for  the  P-3C 
aircraft  for  1979  was  determined  to  be 
104,863.  Then  the  total  aircraft 
support  cost,  less  petroleum  oil  and 
lubricants  (POL)  and  training  support 
was  obtained.  This  cost  is 
$155,257,000.  Annual  flight  hours  were 
divided  into  aircraft  support  costs, 
resulting  in  a  figure  for  cost  per  hour 
without  POL,  which  equates  to  the  cost 
of  a  lab  aircraft,  of  $1,480  per  hour. 
The  C0MNAVAIRPAC  (22)  reported  cost  per 
flight  hour  for  POL  is  $951.  The  cost 
per  flight  hour  is  therefore  $2431.  As 
indicated  before  in  the  assumptions, 
aircraft  depreciation  has  not  been 
factored  into  the  costs. 


TABLE  16 


INSTRUCTOR  COST-  PER  HOUR  FOR 
INSTRUCTIONAL  MEDIA  (in  $) 


OFT/PT 

WST/Aircraft  Lab 

P-3  Flight 

1. 

Pilot 

18 

18 

37 

2. 

NFO 

15 

15 

15 

3. 

FE 

7 

7 

7 

4. 

SS  1/2 

3 

6 

6 

5. 

SS  3 

5 

9 

5 

6. 

COMM 

— 

— 

6 

7. 

ORD 

-- 

3 

5 

As  described  earlier,  the  pilot  and 
flight  engineer  accrue  half  of  the 
aircraft  cost  per  hour  during  Phase  B  of 
the  MCS,  since  they  are  the  only  two 
crew  student  positions  who  log  flight 
hours  during  that  Phase.  During  Phases 
C  and  D,  seven  crew  student  positions 
are  filled.  For  each  flight  hour,  the 
cost  is  distributed  across  all  crew 
positions . 

RESULTS 

Cost  Summaries  for  P-3  Training 
Curriculum 

Table  17  displays  the  cost  per  hour 
for  each  instr uctional  medium  for  each 
crew  position.  The  cost  for  each 
training  device  hour  was  obtained  by 
adding  the  training  device  cost  per 
student  hour  (found  in  table  15)  to  the 
VP-31  instructor  cost  per  hour,  for 
training  devices,  aircraft  lab  and 
aircraft  hours  (from  Table  16). 


Table  18  shows  the  total  cost  for 
each  crew  position.  The  ISD  costs  were 
obtained  from  table  8  and  the  learning 
center  (LC)  costs  from  table  11. 

The  cost  for  each  medium  was 
calculated  by  multiplying  the  costs  per 
hour  (table  17)  by  the  number  of  hours 
each  student  utilized  a  media  displayed 
in  the  master  course  syllabus  (table  2). 

Billet  costs  were  determined  by  that 
proportion  their  scheduled  MCS  training 
time  is  of  a  man-year.  An  average 
student  rank/rate  was  used.  For 
example,  a  first  tour  pilot  is  probably 
a  ENS,  he  takes  100  working  days  to 
complete  the  syllabus.  This  is  about  42 
percent  of  a  man-year.  Koehler  was 
referenced  for  annual  billet  costs 
(15,  16). 

The  costs  to  train  a  typical  P-3 
crew  have  been  summarized  from  Table  18 
and  are  displayed  as  Table  19. 
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TABLE  17 


COST  PER  HOUR  PER  MEDIUM  PER  CREW  POSITION 
(In  $  ) 


P-3 


2C-4 5/ 

MCS 

2F69 

2F87F 

14B44  2F87T 

Phase  B  Phase  C,D 

Pilot 

1st 

Tour 

( P  or 

C)  51 

52 

37  40 

1252 

384 

2nd 

Tour 

(B  or 

C)  51 

52 

40 

1252 

384 

2C-45/ 

Static 

2F69 

2F87F  ; 

2F87T  A/C 

P-3 

FE  (B 

or  C) 

40 

41 

29  UNK» 

122? 

354 

A/C  LAB 

2F69 

2F87T 

i  -3 

NFO  B 

(1st 

Tour ) 

755 

40 

_ 

362 

C 

(1st 

Tour ) 

755 

— 

37 

362 

A/C 

14B44 

2F69 

2F87T  LAB 

P-3 

SS  1/2 

B 

22 

31 

746 

353 

C 

22 

28  746 

353 

A/C 

14B40 

2F69 

2F87T  LAB 

P-3 

SS  3  B 

_ 

34 

749 

353 

C 

^6 

““ 

31  749 

353 

A/C  LAB 

P-3 

ORD  B 

496 

352 

C 

496 

352 

A/C  LAB 

P-3 

COM,  B 

745 

352 

NOTE: 

•  Static  A/C  are 

assumed  to 

have  no 

cost  since  they 

do  not  have 

power 

and 

can  be  any 

A/C  not  in 

use  on  the  flight  line. 

CREW  POSITION 

COST 

TABLE  18 

PER  CREW  POSITION 
(in  $) 

2C45/ 

Pilot 

ISP 

LC 

2F69 

2F87F 

14B44 

2F87T 

P-3 

1st  To  ur 

(ENS) 

653 

837 

1122 

1456 

1 1 1 

1280 

51980 

2nd  Tour 

(LCDR ) 

653 

837 

Billet 

459 

Costs 

1560 

NA 

TOTAL 

1280 

32254 

1 4 i 389 
29,215 
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TOTAL 

$71,828 

$66,257 


TABLE  18  (Continued) 


COST  PER  CREW  POSITION 
(in  $) 

2C45/ 


FE 

ISD 

LC  2F69  2F87F 

P-3 

(AD. 

-3) 

355 

618  320 

1476 

71536 

Billet  Costs 

TOTAL 

6,511 

$80,816 

A/C 

ISD 

LC  LAB 

2F69  2F87  P-3 

NFO 

B 

(ENS) 

866 

661  15855 

2760 

25340 

C 

(ENS) 

866 

661  20385 

2738  19548 

Billet  Costs 

TOTAL 

11,375 

$56,857 

11,375 

$54,573 

A/C 

ISD 

LC  14B44 

2F69  2F87T 

LAB 

SS 

1/2 

B  (AW-3) 

603 

611  12511 

952 

14920 

C  (AW-3) 

603 

611  1188 

896 

26110 

P-3 

Billet  Costs 

TOTAL 

12708 

6411 

$37, 

499 

10590 

6411 

$46, 

409 

ISD 

LC  14B40  2F69 

2F87T 

SS  3 

B 

(AW-3) 

753 

729 

1088 

C 

(AW-3) 

753 

729  2016 

i  — 

992 

Billet 

15Z1/15E16 

A/C  LAB 

P-3 

Costs 

TOTAL 

2016  * 

1  947  4 

24710 

4856 

53626 

-- 

18725 

24710 

4856 

52781 

*  (Device  15Z1  is  obsolescent,  costs  are  not  identified  in  C0C-20 
index,  device  1 4B 4 0  costs  were  therefore  assumed.) 


A/C 

Billet 

ISD 

LC 

LAB 

P-3 

Costs 

TOTAL 

ORD  B  (AO-3) 

188 

883 

18848 

8448 

2063 

30430 

C  (AO-3) 

188 

883 

14800 

8448 

2063 

26382 

(A/C  LAB  for 

B  assumes 

2  students  per 

class,  C 

four  students 

per 

class . ) 

A/C 

Billet 

ISD 

LC 

LAB 

P-3 

Costs 

TOTAL 

COM  B  ( AT — 3 ) 

1017 

615 

37995 

10560 

8238 

58425 

(A/C  LAB  assumes  two  students.) 


TABLE  19 


FRS  TRAINING  COSTS  FOR  A  P-3C  CREW 


Pilot  2nd  Tour 

1st  Tour 

$  66,257 
71,828 

Flight  Engineer 

80,816 

Naval  Flight  Officer  2  0 

54,573 

109, 146 

Sensor  Stations,  1  &  2  2 

6  46,409 

92,818 

Sensor  Station,  3 

52,781 

Ordinance  Man 

26,382 

Total 

$500,028 

DISCUSSION 

A  summary  of  the  percentages  that 
the  various  components,  ISD,  learning 
center,  training  devices,  aircraft,  and 
student  billet  costs  represent  along 
with  their  approximate  costas,  gre 


depicted  below 

.  Table  19, 

which  arrays 

the  data,  has 

been  compiled  from 

approximate  costs  to  train 

a  P-3C  crew, 

about  $500,000 

.  Table  18 

arrays  the 

cost  for  each 

component  by 

crew 

position . 

Percentage 

Component 

$  Cost 

of  Cost 

ISD 

5,540 

1.  1 

LC 

6,448 

1.2 

Training 

Devices 

21,716 

4.3 

P-3C  Total 

375,719 

75 

(Lab  Only) 

(126,515) 

(25) 

(Flight) 

(249,204) 

(50) 

Billet  Cost 

92  .606 

18.5 

Approx . 

$500,000 

The  cost  for  ISP-related  activities 
when  considered  in  the  total  costs  to 
train  a  P-3C  crew  is,  when  viewed  in  the 
total  P-3  program,  minor.  Even  though 
their  possible  impact  on  overall  costs 
are  significant.  This  is,  in  part,  a 
reflection  of  spreading  the  ISD  costs 
across  the  probable  life  cycle  of  the 
weapons  system.  The  learning  center 
costs  are  also  minor  when  viewed  in  the 
total  crew  training  costs,  about  1.2 
percent.  Training  devices  also  only 
represent  4.3  percent  of  the  total 
training  costs,  this  was  a  surprising 
finding  in  view  of  their  initial 
acquisition  costs.  The  P-3  aircraft 
costs  as  related  to  the  crew  training 
costs  are  the  major  contributors.  The 


total  P-3  costs  represent  75  percent  of 
the  cost  to  train  a  P-3C  crew.  The 
flight-hour  costs  represent  50  percent 
of  the  total  crew  costs  and  the  use  of 
the  P-3  aircraft  as  a  lab  represents  25 
percent . 

A  model  specifically  tailored  for 
collection  of  dcta  in  order  to  support 
decision  making  in  fleet  replacement 
squadrons  seems  to  be  warranted,  in  that 
the  data  to  support  such  a  model  is 
existing  and  can  be  used,  if  correctly 
applied,  for  decision  making.  It  should 
al3o  be  noted  that  this  report  only 
covers  the  cost  side  of  a  possible  cost 
benefit  model.  The  benefit  assumptions 
also  should  be  considered  in  a  total 
system  model . 

The  costs  for  the  P-3  when  used  as 
an  aircraft  lab  are  extremely  high  and 
contribute  about  25  percent  of  the  cost 
to  train  a  crew.  The  cost  impact  of  A/C 
lab  is  significant  in  the  NFO,  SS  1/2, 
SS-3,  ORD,  and  CON  B  crew  positions.  A 
cost  analysis  appears  to  be  warranted  in 
order  to  support  decisions  regarding  the 
acquisition  of  supplemental  training 
devices.  (For  example,  if  the  P-3  Lab 
were  to  be  used  by  the  P-3C  NFO  for 
harpoon  missile  training  due  to  a 
paucity  of  realism  in  the  training 
device,  application  of  a  cost  model 
might  support  acquisition  of  a  requisite 
device. ) 


The  fine  tuning  of  the  training 
support  requirements  for  a  major  weapons 
system  is  a  complex  process.  The  costs 
involved  are  not  clear  to  Navy  planners 
since  they  cross  budgetary  lines  of 
control.  A  model  such  as  this  can 
assist  the  planners  in  a  fleet  training 
location  to  help  estimate  the  relative 
impact  on  the  total  training  cost  that 
are  reflected  by  component  erst  changes. 


i 
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ABSTRACT 

Rapidly  increasing  costs,  and  military  requirements  now  dictate  a  total  system  life  cycle 
approach  to  training  system  design  and  development.  Quantitative  evaluation  of  training 
system  concepts  and  acquisition  alternatives  requires  the  use  of  a  rational  procedure  or  model 
which  defines  the  relationships  between  all  elements  of  the  system.  A  first  generation  train¬ 
ing  system  cost  effectiveness  (TSCE)  model  has  been  developed  which  allows  media  designers, 
instructional  psychologists,  and  system  analysts  to  integrate  their  concepts  on  a  total  system 
life  cycle  basis.  This  model  uses  the  Instructional  System  Development  (1SD)  process  to 
derive  media  mix  and  syllabus.  Terminal  learning  objectives  (TLOs)  and  syllabus  are  linked  to 
design-oriented  media  characteristics  and  performance.  Application  of  the  TSCE  model  to 
undergraduate  pilot  training  system  concept  evaluation  provides  a  detailed  understanding  of 
how  the  major  system  variables  interact  in  addition  to  quantitative  definition  of  total  system 
performance. 


BACKGROUND 

This  paper  presents  preliminary  results  of  an 
on-going  research  task  at  the  North  American 
Aircraft  Division,  dealing  with  the  definition  and 
application  of  a  Training  System  Cost  Effective¬ 
ness  (TSCE)  Model  for  evaluation  of  undergraduate 
pilot  training  system  concepts. 

The  continuing  objective  of  this  task  has  been 
to  model  the  entire  Navy  strike  training  system 
and  use  the  TSCE  model  to  evaluate  system  alterna¬ 
tives.  The  training  system  includes  a  curriculum, 
training  media  (aircraft,  simulators,  academic 
materials),  an  integrated  support  system 
(facilities,  equipment,  personnel,  consumables), 
and  a  training  management  system. 

Navy  strike  training  is  presently  taught  in 
two  phases  (intermediate  and  advanced)  which 
require  a  total  of  220  training  days  as  shown 
below  in  Figure  1.  The  Student  Naval  Aviator  (SNA) 
is  awarded  an  Ensign's  comnisaion  at  the  end  of 
Aviation  Officer  Candidate  Training  (AOCT).  The 
SNA  must  then  complete  primary  flight  training  in 
T-34C  aircraft  before  entering  intermediate  strike 
training  in  Rockwell  T-2C  aircraft.  Wings  are 
awarded  on  completion  of  advanced  strike  training 
in  TA-4J  aircraft  followed  by  transition  to  opera¬ 
tional  aircraft  in  Fleet  Replacement  Squadrons 


Figure  1  Navy  Strike  Flight  Training 


(FRS).  The  present  curricula  designations  are 
shown  in  Figure  1  as  CNATRAINST  1542 .27B,  .59A, 
.56,  and  .20C  from  Ref.  (D,  (2),  (3)i 
and  respectively. 

Although  the  TSCE  model  has  been  developed  for 
evaluation  of  various  Navy  strike  pilot  (Inter¬ 
mediate  +  Advanced)  training  system  concepts,  the 
same  general  procedures  are  also  applicable  to 
Navy  Primary  and  FRS  phases  as  well  as  most  other 
flight  training  systems. 

Review  of  the  literature,  discussion  with 
recognized  experts,  and  analysis  of  existing 
models  indicated  in  April  1979  that  no  generally 
accepted  model  or  procedure  existed  for  evaluation 
of  the  entire  training  system.  Additional 
research  provided  the  following  insights: 

1.  All  training  system  designs  should  start 
with  an  analysis  of  the  operational  tasks  to  be 
performed  by  the  (rained  individual.  Certain  of 
these  tasks  are  selected  as  tasks  to  be  trained 
and  training  objectives  or  requirements  should  be 
defined  before  system  design  is  initiated. 

2.  Recent  advances  in  the  application  of 
instructional  psychology  and  modem  instructional 
technology  have  prompted  the  definition  of  a 
systematic  design  procedure  for  instructional 
systems  called  ISD  which  begins  as  described  in 
item  1.  above.  When  the  Instructional  System 
Development  (ISD)  process  is  used,  a  direct  link 
is  established  between  training  objectives,  media 
mix,  and  curriculum. 

3.  Existing  versions  of  the  ISD  process 
define  hands-on  (aircraft,  and  simulator)  media 
requirements  and  capabilities  in  general  terms 
which  do  not  provide  adequate  guidance  for  media 
design. 

4.  It  became  clear  that  computer  processing 
of  data  associated  with  the  ISD  and  other  TSCE 
submodels  would  be  necessary  in  order  to  dras¬ 
tically  reduce  iteration  time  and  increase 
responsiveness  of  the  entire  TSCE  model  to  the 
training  system  design  process. 
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5.  The  design  of  pilot  training  ayateas  ia 
atill  partially  an  art  due  to  the  coaplexities  and 
psychological  aubtleties  of  some  training  system 
variables.  Since  relevant  experimental  or  ana¬ 
lytic  data  are  sparse,  training  system  analysis 
and  design  are  still  highly  dependent  on  extra¬ 
polations  froa  historical  data  and  statistically 
analyzed  expert  opinion. 

THE  TSCE  MODEL 

The  TSCE  model  has  been  organized  into  five 
basic  modules  as  shown  in  Figure  2.  Each  module 
contains  one  or  more  submodels.  The  Input  Module 
includes  all  system  requirements,  criteria,  and 
guidelines  initially  defined,  such  as: 

.  Existing  Assets  (Facilities,  Media, 
Equipment) 

.  Training  Policies  and  Doctrine 
.  Terminal  Learning  Objectives  (TLOs) 

.  Pilot  Training  Rate  (PTR) 

.  System  Life  and  IOC 
.  Student  and  Aircraft  Attrition  Data 
.  Existing  Curriculum 
.  Media  Characteristics  Data 
.  Organization  and  Basing 
.  Personnel  Data  and  Costs 
.  Training  Management  System  (TMS)  Functions 
and  Configuration 


Figure  2  Organization  of  the  TSCE  Model 

The  above  listing  is  representative  but  not 
exhaustive.  The  Input  Module  also  contains 
initial  media  design  concepts  and  integrated 
logistic  support  (ILS)  concepts  to  be  evaluated. 

The  Instructional  System  Module  includes  the 
ISD  submodel,  ASSEM  (Aircraft/Simulator  Syllabus 
Extrapolation  Model),  and  the  TMS  submodel  as 
shorn  in  Figure  3.  Terminal  Learning  Objectives 
(TLOs)  are  decomposed  in  the  ISD  submodel  to  a 
hierarchy  of  enabling  objectives  (EOs)  by  a 
subject  Blatter  expert  (SME)/inatructional  psychol¬ 
ogist  (IP)  team.  This  process  establishes  the 
master  sequence  of  instruction.  All  EOs  are 
stated  in  behavioral  terms  on  objective/media 
worksheets  along  with  their  associated  conditions 
and  standards,  and  classified  as  academic  or 
hands-on  (simulator,  aircraft)  objectives.  The 
same  SHE/ IP  team  then  establishes  media  require¬ 
ments  for  each  EO  by  selecting  those  siedia 
attributes  which  assure  effective  training  and 
efficient  instruction.  The  media  attributes  list 
for  academic  EOa  is  shown  in  Figure  4  and  the 
hands-on  attributes  are  shown  in  Figure  }.  The 
hands-on  attributes  contain  an  equipment  list 


Figure  3  Instructional  System  Module 

which  is  relevant  to  aircraft  or  simulators,  an 
aircraft  characteristics  list  organized  in 
discrete  performance  levels,  and  a  simulator 
characteristics  list  which  includes  most  of  the 
critical  design  criteria  for  student  and  instruc¬ 
tor  stations.  These  attribute  lists  must  be 
defined  with  care.  In  particular,  the  hands-on 
attributes  should  be  design-oriented  and  suffi¬ 
ciently  discriminating  to  represent  significant 
differences  in  training  effectiveness  and  cost  of 
media  design  concepts.  A  particular  EO  may 
require  visual  cues  for  effective  training  but  the 
characteristics  (field  of  view,  B&W  or  color, 
night/dusk  or  daylight)  and  sophistication  of 
those  visual  cues  should  also  be  addressed, 
depending  on  the  EO.  At  this  stage  in  the  ISD 
submodel,  insertion  of  design-oriented  attributes 
improves  the  media  selection  process  as  long  as 
selection  of  alternatives  can  be  resolved  and 
decisions  are  not  inhibited.  Training  objectives 
must  be  linked  with  principal  design  attributes  of 
the  media  in  this  manner  before  changes  in  media 
can  be  traced  to  changes  in  curriculum,  media 
quantities,  and  system  life  cycle  cost. 
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Media  Selection 

The  initial  step  of  media  selection  within  the 
ISO  submodel,  occurs  when  media  requirements  from 
the  objective/media  worksheets  are  compared  with 
media  capabilities  for  each  EO  in  the  objective 
hierarchy.  Media  capabilities  are  defined  in 
matrix  format  (attributes  versus  media  candidates) 
using  the  same  list  of  media  attributes  shown  in 
Figures  4  and  5.  A  team  of  SMEs,  IPs,  and 
instructional  technologists  (ITs)  rate  all 
candidate  media  against  the  attributes  with  a 
value  scale.  Comparison  of  media  requirements 
with  media  capabilities  allows  the  media  to  be 
scored  for  each  EO.  Only  those  media  which  meet 
*ii  requirements  to  some  degree,  are  considered 
acceptable  for  each  EO.  Those  candidates  not 
meeting  all  requirements  are  dropped  for  that  EO. 
Media  requirements  for  all  EOs  are  processed  in 
this  manner  and  acceptable  media  are  then  ranked 
according  to  the  number  of  EOs  satisfied.  These 
acceptable  media  are  then  considered  in  combina- 
tions  of  2 ,  3,  4....n  at  a  time  and  ranked,  in 
terms  of  EOs  satisfied  by  media  mix.  Those  media 
mixes  which  satisfy  all  EOs  are  then  evaluated  in 
terms  of  relative  life  cycle  cost  using  prelim¬ 
inary  estimates  of  media  quantities,  and  a  media 
mix  selection  is  made.  Hands-on  EOs  associated 
with  the  selected  media  mix  are  then  listed  by 
training  medium  and  preliminary  DPE  (demonstra¬ 
tion,  practice,  evaluation)  time  increments  are 
assigned  to  each  of  the  media  in  the  mix,  by  EO. 
The  selected  media  mix,  identification  of  EOs  by 


medium,  assignment  of  time  increments  by  EO/media 
combination,  and  sequence  of  instruction  from  the 
objective  hierarchy  provide  the  principal  input 
data  for  development  of  a  training  syllabus.  All 
of  these  data  generated  in  the  ISD  submodel  are 
loaded,  when  available,  into  a  computerized  data 
base  which  facilitates  each  successive  stage  of 
synthesis. 

Curriculum  Development 

Syllabus/Curriculum  development  begins  with 
computer  interactive  sequencing  of  all  handa-on 
objectives  and  assignment  of  a  master  sequence 
number  code.  Academic  objectives  are  then  grouped 
as  prerequisites  to  hands-on  objectives  where 
applicable,  assigned  a  sequence  number,  and 
further  grouped  into  lessons.  A  preliminary 
syllabus  is  available  at  this  stage  which  includes 
a  media  hours  summary,  instructional  sequence,  and 
lesson  groupings.  The  syllabus  is  then  expanded 
to  a  curriculum  with  the  development  of  training 
module  schedules  and  total  contact  time  data.  A 
final  computer  interactive  inspection  and  edit  is 
completed  prior  to  printout  of  the  integrated 
curriculum. 

Curriculum  Validation 

Since  the  curriculum  influences  moat  aspects 
of  training  system  design  including  system 
effectiveness  and  life  cycle  cost,  validation  of 
the  curriculum  is  initiated  as  soon  as  a 


159 


preliminary  ayllabus  (distilled  curriculum)  is 
available,  and  proceeds  in  parallel  with  training 
system  design  and  implementation  until  total  vali¬ 
dation  has  been  completed.  Validation  examines 
correspondence  of  the  ISD-generated  curriculum  and 
its  components  to  perceived  reality  in  accordance 
with  the  three  steps  listed  below: 

1.  Theoretical  validity  (assessed  prior  to 
training  system  design) 

.  Theory  underlying  the  ISO  process. 

.  Assumptions  (stated  or  implied) 

•  Transition  from  perceived  reality  to  the 
ISO  submodel  used. 

2.  Data  validity  (assessed  during  training 
system  design) 

.  Accuracy,  completeness,  impartiality,  and 
appropriateness  of  the  original  input  data. 

.  How  the  ISO  submodel  deals  with  transforma¬ 
tion  of  the  original  input  data. 

3.  Operational  validity  (assessed  during 
training  system  operation) 

.  Divergence  between  the  ISD-generated 
curriculum  and  the  curriculum  which 
evolves  from  training  system  operation. 


The  ASSEM  process  was  developed  from  concepts 
published  in  Ref.  (5)  and  includes  the  seven 
basic  steps  listed  below: 

.  Since  syllabus  extrapolation  in  ASSEM  is 
based  on  the  weighted  application  of 
performances  differences  between  media  of 
the  same  type,  syllabus  differences,  due  to 
other  effects  such  as  non-equivalence  of 
TLOs  and  media  types  must  be  eliminated 
prior  to  syllabus  extrapolation.  The 
initial  step  is  therefore  to  define  coamon 
and/or  equivalent  TLOs  between  the  existing 
and  candidate  syllabi  as  shorn  in  Figure  6. 

.  Organization  of  common  TLOs  and  associated 
media  hours  by  training  mission  or  stage 
(completed  by  SME/IF  team). 

.  Derivation  of  simulator/aircraft  training 
transfer  factors  for  specific  media/mission 
combinations  (from  existing  data  and  by  SME 
team) . 

.  Define  total  hands-on  hours  in  terms  of  one 
training  medium  (so  that  syllabi  can  be 
compared)  by  using  training  transfer 
factors  to  convert  all  simulator  hours  to 
aircraft  hours  (or  the  converse  for 
simulator  syllabus  extrapolation). 


The  initial  step  in  this  validation  process 
takes  place  within  the  TSCE  model.  Starting  with 
essentially  the  same  inputs  and  in  parallel  with 
the  ISD  submodel,  a  second  (non-ISD)  syllabus  is 
derived  by  extrapolation  from  the  existing 
training  command  syllabus  using  che  Aircraft/ 
Simulator  Syllabus  Extrapolation  Model  (ASSEM) 
shown  in  Figure  6.  Comparison  of  the 
ASSEM-derived  hands-on  syllabus  with  the 
ISD-derived  hands-on  ayllabus  provides  an  initial 
check  on  the  media  hours  by  training  stage  listed 
in  the  ISD  syllabus. 


.  Derive  weighting  factors  for  each  media 
characteristic/training  mission  combination 
using  SME  team. 

.  Derive  training  rate  factors  (TRFs)  for 
each  of  the  candidate  medium  performance 
characteristics  and  ratio  to  the  TRF  for 
the  existing  or  baseline  medium. 

.  Run  the  ASSEM  computer  program  which 

calculates  the  product  in  matrix  format  of 


Figure  6  Aircraft/Simulator  Syllabus  Extrapolation  Model 
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weighting  factor,  TRF  ratio,  and  existing 
syllabus  hands-on  hours  by  training  mission 
to  yield  extraplated  syllabus  hands-on 
hours.  This  extrapolated  hands-on  syllabus 
is  then  compared  with  the  candidate  or 
ISD-generated  syllabus  as  shown  in  Figure  6. 

The  ASSEM  submodel  is  based  on  the  following 
three  assumptions: 


1.  The  skill  level  attained  by  a  student 
aviator  is  a  function  of  training  time  as  defined 
in  Figure  7. 


2.  The  relationship  between  media  design 
characteristics  (sustained  load  factor,  field  of 
view,  etc.)  and  training  mission/stages  (famil¬ 
iarization,  basic  instruments,  etc.)  can  be 
adequately  quantified  by  a  team  of  expert  aviators 
and  statistically  analyzed  to  provide  weighting 
factors. 


3.  Syllabus  media  hours  are  inversely  pro¬ 
portional  to  media  characteristics  where  the 
proportionality  is  assumed  to  be  linear,  except  in 
those  cases  where  the  team  of  experts  establish  a 
different  relationship. 

The  curve  in  Figure  7  can  be  approximated  by 
three  linear  segments.  The  two  horizontal 
segments  represent  transition  and  saturation, 
respectively,  and  the  linear  segment  connecting 
them  represents  a  constant  learning  rate.  During 
the  initial  or  transition  portion  of  the  curve, 
skill  level  is  not  increased  significantly  even 
though  training  time  is  expended  for  general 
familiarzation  (getting  confortable)  with  the 
aircraft  or  simulator.  ASSEM  uses  a  statistical 
relationship  for  calculation  of  transition  time. 

Derivation  of  the  media  characteristic  versus 
training  mission  weighting  factors  has  been 
completed  by  a  team  of  expert  aviators  which 
included  Navy  trained  test  pilots,  an  Air  Force 
fighter /attack  pilot,  aeronautical  engineers  with 
civilian  piloting  experience,  and  an  educational 
psychologist  with  instructor  pilot,  simulator 
development,  and  human  factors  experience.  The 
mean  values  of  individual  scores  for  aircraft  are 
recorded  in  matrix  format  as  shown  in  Figure  8. 

The  same  type  of  matrix  has  been  defined  for 
simulators.  Weighting  factors  have  been  developed 
from  these  means  which  quantified  the  team's 
evaluation  of  the  relative  importance  of  each 
performance  characteristics.  For  each  mission, 
the  sum  of  the  weighting  factors  is  1.0. 

Existing  syllabus  hours,  the  weighting  factors 
described  above,  and  training  rate  factors 
described  previously  are  the  three  types  of  data 
used  in  the  ASSEM  computer  program.  Training  rate 
factors  (TRFs )  are  defined  as  the  quotient  of  each 
specific  training  medium  performance  character¬ 
istic  to  the  equivalent  characteristic  of  the 


\CH  A  R  ACT  E  R  1ST  ICS 

TRAINING 

MISSIONS 

MANEUVERABILITY 

SERVICE 
i  CEILING 
(100  FPM) 

SPEED  REGIME 

m 

MAX. 
RANGE 
AT  SL 

RATE  OF  CLIMB 
AT  SL 

AVIONICS 

AND 

SYSTEMS 

SOPHISTI¬ 

CATION 

POWER 

APPROACH 

CONTROL 

INSTANT. 

LOAD 

FACTOR 

SUSTAINED 

LOAD 

FACTOR 

I 

ROLL  ACCEL: 
BANK  ANGLE 
IN  1ST  SEC 

MAX. 

LEVEL 

FLIGHT 

SPEED 

LANDING 

APPROACH 

SPEED 

ENGINE 

OUT 

ALL 

ENGINES 

GUST 

RESPONSE 

FAMILIARIZATION 

2.3 

1.8 

2.3 

■a 

1.8 

3.5 

1.1 

2.5 

mm 

mm 

2.5 

BASIC  INSTRUMENTS 

1.0 

0.8 

2.0 

mm 

1.3 

2.0 

1.5 

HI 

1.9 

2.0 

2.4 

RADIO  (T.O.  AND  U 
INSTRUMENTS 

■a 

0.1 

1.4 

1 

3.6 

2.5 

2.3 

D 

1.9 

3.8 

3.8 

FORMATION 

2.6 

2.4 

3.0 

ma 

1.0 

2.6 

1.8 

0.1  1 

I 

2.9 

0.4 

1.5 

NIGHT 

FAMILIARIZATION 

1.0 

1.0 

1.5 

■a 

KB 

■a 

1.4 

,4 

2.3 

2.6 

AIRWAYS  NAVIGATION 

0.1 

0.1 

0.5 

■ 

|  2.5  ! 

0.8 

1.4 

3.5  j 

1.0 

1.6 

3.9 

1,4 

OPERATIONAL  (LOW 
LEVEL)  NAVIGATION 

■a 

2.1 

3.4 

■a 

D 

0.1 

3.8 

□ 

1.8 

2.3 

kb 

AIR  TO-AIR  GUNNERY 

■a 

4.3 

4.0 

3.5 

0.0 

2.9 

2.1 

• 

3.1 

3.9 

0.0 

AIR  COMBAT 
MANEUVERING 

! 

49 

4.9 

4.5 

a 

1 

3.4 

0.8 

4.6 

KB 

AIR  TOGROUNO 
WEAPONS 

4.6 

3.4 

4.4 

■a 

3.3 

II 

3.3 

2.5 

'  ' 

D 

3.9 

KSM 

CARRIER 

QUALIFICATION 

B 

1.0 

3.9 

. 

B 

1 

m 

D 

2.8 

2.8 

MM 

RATE  FROM  ZERO  TO  FIVE  IN  WHOLE  NUMBERS.  WHERE  0  ■  TOTALLY  UNIMPORTANT 

S  -  ABSOLUTELY  ESSENTIAL 


Figure  8  Mean  Values  of  the  Individual  Scores  (For  Aircraft) 
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baseline  medium.  The  TRF,  measures  the  influence 
of  each  medium  characteristic  on  the  time  required 
to  achieve  a  specific  skill  level.  For  most 
characteristics  and  training  missions,  this 
relationship  is  assumed  to  be  linear  as  shown  in 
Figure  9.  The  team  of  experts  specify  a  maximum 
value  (C)  for  each  of  the  media  characteristics. 
These  maximums  represent  values  of  the 
characteristics  above  which  there  is  no  increase 
in  training  capability. 


Figure  9  Relationship  Between  Training  Rate 
Factor  (TRF)  and  Training  Medium 
Performance 

The  TRF  versus  performance  characteristic 
relationship  does  assume  a  non-linear  nature  for 
certain  characteristic/mission  combinations  which 
are  approximated  by  a  piecewise  linear  relation¬ 
ship  also  shorn  in  Figure  9.  The  team  of  experts 
select  break-point  (A,  B)  on  the  piecewise  curve 
and  statistical  analysis  is  applied  to  the 
selected  values. 


Except  for  transition  hours,  the  entire  calcu¬ 
lation  process  for  aircraft  is  shown  pictorially 
in  Figure  10.  In  this  figure  weighting  factors 
obtained  from  Table  I  are  multiplied  by  the  ratio 
of  TRFs  from  Table  II  to  yield  the  values  listed 
in  Table  III  for  each  intersection  of  that 
matrix.  These  values  are  summed  horizontally  to 
obtain  training  effectiveness  factors  by  mission 
for  Aircraft-A  as  listed  in  Column  (IV).  Syllabus 
hours  by  mission  for  baseline  Aircraft-B  are 
listed  adjacently  in  Column  (V).  The  reciprocal 
values  of  Column(IV)  are  then  multiplied  by  the 
baseline  or  existing  syllabus  hours  for  Aircraft-A 
which  are  listed  in  Column  (VI).  These  syllabus 
hours  by  mission  (Columns  V  and  VI)  are  sumned 
vertically  over  all  missions  to  obtain  the  total 
training  time  for  each  aircraft.  In  block  (VII), 
total  training  time  for  the  baseline  Aircraft-B  is 
divided  by  the  total  training  time  for  Aircraft-A, 
to  compute  the  training  effectiveness  for 
Aircraft-A.  This  calculation  process  illustrated 
in  Figure  10,  corresponds  to  the  constant  rate 
portion  of  the  generalized  learning  curve  shown  in 
Figure  7. 

Hands-on  media  hours  extrapolated  from  the 
existing  syllabus  using  ASSEM  as  just  described, 
are  then  compared  with  ISD  hands-on  syllabus  hours 
by  training  mission  as  previously  shown  in  Figure 
6.  If  there  is  reasonable  agreement  (+5%)  between 
syllabi,  the  ISD  syllabus  is  used  as  input  to  the 
TMS  submodel  and  the  Resource  Requirements  Module 
of  the  TSCE  Model.  Otherwise,  ASSEM  and  the  ISD 
Submodel  are  iterated  until  agreement  is  reached. 

TMS  Submodel 

The  Training  Management  System  (TMS)  Submodel 
has  been  shown  previously  in  Figure  3  as  the  final 
part  of  the  Instructional  System  Module.  As  such, 
it  is  the  focal  point  for  integration  of  all 


Figure  10  ASSEM  Calculation  (Example  for  Aircraft) 
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Madia  Hours 


+  "K"  (Hrs) 


+  *M  *  Time  to  Train 


Total  Prepare,  Study 

Contact  Brief/Debrief 

Time  Preflight,  Start, 

Taxi 


•  Weather 

•  Unsatisfactory  Events 

and  Associated  Delays 

•  Medical  Groundings 

•  Plight/Simulator  Event 

Cancellations  Due  to  Instructor 
and/or  Equipment  Not  Available 

•  Other 


Figure  11  Training  Management  System  (IMS)  Submodel 


scheduling  and  personnel  requirements  associated 
with  the  application  of  training  system 
resources.  Within  the  TSCE  Model,  it  sisiulates 
the  effect  of  the  actual  TMS  on  the  training 
system.  A  simplified  version  of  the  TMS  Submodel 
is  shown  in  Figure  11.  The  core  of  this  submodel 
is  a  scheduling  algorithm  which  is  used  to  define 
tradeoffs  between  the  following  system  variables: 

.  Student  attrition  versus  diagnostic  and 
remediation  time 

.  Instructor  scheduling  versus  student 
scheduling 

.  Student  scheduling  by  media  versus  media 
utilisation 

.  Media  utilisation  versus  media  maintenance 
scheduling 

These  trade  studies  provide  the  representative 
list  of  outputs  shown  below: 

1.  Student  schedule  and  remediation  plan  by 
student  performance  profile. 

2.  Changes  (if  any)  in  student  attrition 
policy. 

3.  Instructor  scheduling  by  PTR,  student 
performance  profile,  and  training  medium. 


4.  Schedule-optimised  changes  in  personnel 
allocations  by  organisation,  base,  function,  and 
skill  level. 

5.  Media  utilisation  by  student  performance 
profile  and  remediation  plan. 

6.  Maintenance  scheduling  by  training  medium 
for  various  utilisation  levels. 

7.  Detailed  training  and  associated 
maintenance  schedules  by  student  performance 
profile. 

8.  Estimated  student  attrition  by  student 
performance  profile  and  reflation  plan. 

9.  Change  in  t  (refer  to  Figure  11)  for 
various  scheduling  optimisation  criteria  and 
support  concepts. 

10.  Curriculum  maintenance  scheduling  based  on 
student  response,  media  updates,  fleet  deficiences 
and  changes  in  administrative  procedures. 

The  ISO  Submodel  and  AS SIM  provide  madia  mi>, 
student  curriculum,  and  instructor  curriculum! 
inputs  to  the  TMS  Submodel.  Preliminary  madia 
utilisation  estimates  and  support  concept 
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characteristics  are  obtained  from  the  Resource 
Requirements  Module.  The  TMS  Submodel  provides 
achieved  PTR  and  time-to-train  data  to  the  System 
Evaluation  Module  for  use  in  training  system 
efficiency  calculations.  Detailed  training  and 
maintenance  schedules,  changes  in  system  personnel 
characteristics,  and  TMS  characteristics  are  also 
routed  back  to  the  Resource  Requirements  Module 
for  system  life  cycle  cost  calculations. 

Resource  Requirements  Module 

A  simplified  version  of  this  module  is  shown 
in  Figure  12.  Logistic  Support  Analysis, 


maintenance  schedules  (from  the  TMS  submodel), 
media  RAM  characteristics,  initial  maintenance  and 
non-maintenance  personnel  allocations,  and  pre¬ 
liminary  media  quantities  are  required  inputs  to 
the  simulation.  The  output  from  the  simulation  is 
a  family  of  O&S  trade  data  which  relate  media 
utilization  and  quantities,  personnel  allocations, 
spare  parts,  and  GSE  to  O&S  cost. 

The  acquisiton  cost  analysis  starts  with  the 
preliminary  definition  of  system  acquisition 
alternatives  based  on  state-of-the-art  versus 
advanced  technology  segregation  of  training  media 
and  TMS  components.  Preliminary  media  quantities 


FROM  ISDN 
SUBMODEL 
AND 
ASSEM 


•  Media  Mix 

•  Student 

Curriculum 

•  Instructor 

Curriculum 


INTEGRATED\ 
LOGISTIC 
SUPPORT 
CONCEPT, 


SUPPORT  CONCEPT 
PRELIM  UTILIZATION 


PRELIM  MEDIA] 
AVAILABILITY  I- 
&  UTILIZATION! 

r 

MEDIA  R&M  1 

characteristics! 

E 

LOGISTIC 
SUPPORT 
ANALYSIS 


PRELIM  MEDIA 
QUANTITIES 


•  Planning  Factors 

•  Sqn.  Manning 

I  •  Supplier  Estimates 


|  SEGREGATION  | 
OFSOA 
I  VS  ADVANCED  | 
|  TECHNOLOGY 
•  Media 
e  TMS 


MAINTENANCE] 

PERSONNEL 

ANALYSIS 

•  Quantity 

•  Type 


NON-MAINT 

PERSONNEL 

ANALYSIS 


DETAILED 
O&S  COST 
SUMMARY 


(OPERATIONS! 

AND 

SUPPORT 

(O&S) 

I  simulation! 


O&S 

TRADES 


PERSONNEL 
UNIT  COSTS 


1  FINAL 

TRAINING 

SELECTION 

SYSTEM 

OF  MEDIA 

LCC 

|QUANTITII 

SUMMARY 

ACQUISITION 

COST 

ANALYSIS 


ACQUISITION 

TRADES 


PRELIM  ANALYSIS! 
OF  SYSTEM 
ACQUISITION 
ALTERNATIVES 


DETAILED 

(ACQUISITION 

COST 

SUMMARY 


TO 
SYSTEM 
EVAL 
MODULE/ 


Figure  12  Resource  Requirements  Module 


calculation  of  training  media  quantities,  and 
calculation  of  life  cycle  costs  all  occur  within 
the  Resource  Requirements  Module.  The  ISD  Sub¬ 
model  and  ASSEM  provide  media  mix,  student 
curriculum  and  instructor  curriculum  inputs  to  the 
Module.  The  integrated  logistic  support  concept 
to  be  evaluated,  provides  inputs  for  the  Logistic 
Support  Analysis  which  in  turn,  provides  defini¬ 
tion  of  media  reliability  and  maintainability 
(R&M)  characteristics,  level  of  repair,  initial 
estimates  of  maintenance  personnel,  general 
support  equipment,  training,  spares/repair  parts, 
and  publications.  Most  of  these  items  are  routed 
as  input  data  to  an  Operations  and  Support  (O&S) 
Simulation.  This  event-oriented  simulation  tracks 
hands-on  media  (aircraft  and  simulators)  through 
all  daily  events  including  operation,  servicing, 
and  maintenance  which  would  normally  occur  in  a 
training  environment.  Detailed  training  and 


and  characteristics,  supplier  cost  estimates  s<V 
computerized  historical  regression  submodels  C-uch 
as  DAFCA  III,  IALCCM,  and  MLCCM)  are  used  to  pro¬ 
vide  acquisition  cost  tradeoffs  with  quantities, 
risk,  and  schedule.  Final  selection  of  media 
quantities  is  based  on  O&S  trades,  acquisition 
trades,  and  optimized  scheduling  inputs  from  the 
TMS  submodel.  Feedback  of  revised  media  quan¬ 
tities  to  the  O&S  simulation  and  acquisition  cost 
analysis  allow  final  detailed  O&S  costs  and 
acquisition  coats  to  be  calculated.  These  costs 
are  susnarized  in  life  cycle  cost  format  and 
become  an  input  to  the  System  Evaluation  Module. 

System  Evaluation  Module 

This  Module  combines  outputs  from  the  Instruc¬ 
tional  System  Module  (effectiveness)  and  the 
Resource  Requirements  Module  (cost)  in  a  variety 
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of  evaluation  formats  which  allow  additional 
system  alternatives  to  be  defined,  or  a  recom¬ 
mended  system  to  be  seLected.  The  following 
criteria  are  used  for  system  selection  in  the 
order  shown  below: 


1.  Training  Effectiveness  •  Percent  EOs 
completed  to  the  conditions  and  standards 
specified. 


E 


(EOs  completed  -  1505) 
(Total  EOs  =  1530)  x 


98.4% 


2.  Pilot  Training  Rate  •  Percent  of  PTR  goal 
or  requirement  achieved  with  total  EOs/student 
completed 


recommended  system  is  selected  and  described  as 
listed  below: 

.  Printout  of  all  TLOs /EOs 
•  Curriculum  (Student,  Instructor) 

.  Media  Mix  (Characteristics,  Configurations) 
.  TMS  Characteristics  and  Configuration 
(Hardware,  Software,  Communications ) 

.  System  Life  Cycle  Cost 
.  System  Alternatives  Evaluation  Summary 
.  System  Implementation  Plan 
.  Preliminary  Specifications  for  Training 
Media 

.  Integrated  Logistics  Support  Plan 
CONCLUSIONS 


(Achieved  PTR  -  520  Pilota/Yr.)  .  „  * 

(PTR  Goal  or  -  570  Pilota/Yr.))  “ 

Requirement 


3.  Training  System  Cost 
.  System  LCC 

.  Average  annual  O&S  cost  per  student 
for  aircraft  simulators,  and  academics 

4.  System  Cost  Effectiveness 

.  Aircraft  O&S  cost/Aircraft  EO 
completed 

.  Simulator  O&S  Cost/Simulator  EO 
completed 

.  Academic  O&S  Cost/Academic  EO 
completed 


5.  Training  System  Efficiency 


TLOs  or  EOs 
“  Time-to-Traln 


and 


Eff  ,  Achieved  PTR 
'  ”  Annual  O&S  Cost 


Each  evaluation  of  a  training  system  alterna¬ 
tive  provides  values  for  all  of  the  above  selec¬ 
tion  criteria.  These  values,  when  compared  with 
previous  alternatives  (baseline,  existing  system) 
may  prompt  the  definition  of  additional  system 
alternatives  for  evaluation  which  generate  changes 
in  any  or  all  of  the  three  basic  modules  (Input, 
Instructional  System,  Resource  Requirement)  of  the 
TSCE  Model  as  shown  in  Figure  2 . 

When  training  effectiveness,  PTR,  and  train¬ 
ing  system  efficiency  have  been  maximized  at  the 
lowest  possible  system  life  cycle  cost,  a 


This  paper  has  described  the  Rockwell  TSCE 
Model  in  its  first  generation  of  definition  and 
application.  It  is  both  a  complex  and  hybrid 
(manual  and  automated)  process  which  represents  a 
complex  real  world  training  system.  Architecture 
of  the  Model  is  expected  to  evolve  with  continuing 
application  and  validation.  Additional  automation 
is  planned  for  the  future  which  will  facilitate 
general  purpose  application  of  the  Model  to  most 
training  systems.  Use  of  the  TSCE  Model  expedites 
training  system  integration  and  provides  a  trace¬ 
able  rationale  for  system  selection.  Insertion  of 
hands-on  media  design-oriented  characteristics  in 
the  ISD  Submodel  has  proven  to  be  a  useful  inno¬ 
vation  for  linking  the  media  design  process  to 
training  objectives  and  hence  to  training  effec¬ 
tiveness  . 
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THE  RELEVANCE  OF  COGNITIVE  PSYCHOLOGY 
TO  INSTRUCTIONAL  TECHNOLOGY 


Gary  G.  Miller 
Instructional  Psychology 
Northrop  Corporation 


This  paper  discusses  the  recent  shift  in  emphasis  in  psychology  from  behaviorism  to 
cognitive  psychology  and  the  relevance  of  this  development  to  the  field  of  instructional 
technology.  Several  problem  areas  in  the  application  of  instructional  psychology  are 
discussed  including,  (a)  the  development  of  prescriptive  handbooks/guidelines  and  (b) 
a  preoccupation  with  hardware  technology  at  the  expense  of  instructional  processes.  Recent 
research  is  cited  which  suggests  that  the  basis  for  many  of  the  Instructional  designs 
Incorporated  in  DoD  and  Industry  wide  training  systems  may  not  be  as  firm  as  once  thought. 
Adoption  of  some  of  the  recent  findings  in  cognitive  psychology  and  a  union  of  behavior¬ 
ism  and  cognitive  psychology  are  suggested  as  a  partial  solution  of  some  of  the  above 
identified  problem  areas.  Some  of  the  advantages  of  adopting  a  decision  oriented  approach 
to  the  design  of  training  systems  are  also  discussed. 
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As  a  research  psychologist  for  the  Air 
Force,  I  am  confronted  regularly  with  the  re¬ 
quirement  to  develop  tools  by  translating 
what  Is  known,  particularly  In  the  design  of 
learning  environments  utilizing  simulation, 
and  delivering  these  tools  In  easy-to-use 
cook-book  formats  for  use  by  the  Air  Force 
training  community.  Those  of  us  who  refer  to 
ourselves  as  Instructional  technologist  have 
often  found  this  enterprise  to  be  more  diffi¬ 
cult  than  anticipated.  Before  undertakings 
such  as  developing  step-by-step  procedures 
for  the  design  of  instructional  environments 
can  even  have  a  glimmer  of  potential  success, 
certain  basic  assumptions  must  be  met.  Among 
these  assumptions  are  (a)  instructional  sci¬ 
ence  is  sufficiently  developed  to  the  extent 
that  accurate  predictions  can  be  made  rela¬ 
tive  to  the  Instructional  environment,  and  (b) 
what  is  known  is  sufficiently  general izable 
that  it  can  be  applied  in  diverse  situations. 
What  I  hope  to  show  in  this  paper  is  that 
instructional  science  has  a  long  way  to  go 
before  accurate  predictions  can  be  made  rela¬ 
tive  to  the  specification  of  instructional 
environments  and  that  many  of  the  rules/ 
principles  of  learning  that  have  been  the 
cornerstones  of  our  thinking  could  be  more 
aptly  classified  as  exceptions  rather  than 
rules.  As  an  alternative,  I  will  discuss 
the  potential  advantages  of  adapting  a  student- 
centered  cognitive  perspective  at  the  expense 
of  our  oftimes  preoccupation  with  applying 
educational  technology  to  an  educational  sys¬ 
tem  that  has  not  changed  demonstrably  since 
the  days  of  Aristotle. 

Some  Background 

Perhaps  a  look  at  where  the  field  of  in¬ 
structional  technology  emerged  might  shed 
some  light  on  our  current  situation.  During 
World  War  II,  an  active  program  of  research 
and  application  in  training  involving  program¬ 
med  Instruction  and  training  devices  offered 
some  promise  that  the  behaviorist  learning 
principles  of  Guthrie  and  Skinner  could  be 
applied  In  military  training.  This  promise 
intensified  In  the  post-Sputnlk  era  as  psy¬ 
chologists  were  encouraged  to  apply  technology 
and  the  systems  approach  to  the  Improvement 
of  training  and  education.  A  new  generation 
of  applied  psychologists,  were  trained  to 
apply  modern  technology  to  the  design  of  learn¬ 
ing  environments.  As  a  basic  structure,  the 
systems  approach  was  lifted  directly  from 
weapons  system  development  and  became  known  as 
Instructional  Systems  Development  (ISD)  in  the 
military.  ISD  worked  quite  well  for  such 
activities  as  analyzing  system  requirements  and 
developing  objectives  and  tests.  However, 
when  it  came  to  the  nuts  and  bolts  activities 
of  designing  and  developing  Instruction,  in¬ 
structional  technologists  found  themselves  In 
the  position  of  pledging  allegiance  to  theories 
in-vogue  at  the  time,  primarily  those  involv¬ 
ing  behaviorist  principles  of  learning.  My 
contention  is  that  the  field  of  instructional 


technology  has  not  only  adopted  these  princi¬ 
ples  but  has  cast  them  in  concrete!  For  the 
past  twenty  years  in  military  instructional 
technology  we  have  been  concerned  with  the 
application  of  modern  technology  to  delivery 
systems  for  instruction  with  relatively  little 
interest  in  the  process  of  learning  and  in¬ 
struction.  As  a  result  of  this  preoccupation 
with  the  technology  delivery  system  and  lack 
of  emphasis  on  learning  process  many  of  us  are 
beginning  to  agree  with  Liefer(l)  that 
learning  appears  to  be  determined  more  by  what 
is  delivered  rather  than  the  delivery  system. 

In  spite  of  much  evidence  today,  the  military 
training  community  still  places  little  formal 
recognition  on  the  instructional  features  of 
training  environments,  particularly  those  in¬ 
volving  training  equipment.  The  primary  con¬ 
sideration  is  still  hardware,  media,  and  the 
delivery  system.  Perhaps  we  should  take  a 
step  back  at  this  point  to  look  at  whether 
instructional  science  has  matured  sufficiently 
to  permit  "psychological  engineering"  of  basic 
research  into  application. 

The  Decline  and  Fall  of  learning  Theories 

There  is  little  doubt  that  Skinner's 
principles  have  been  adopted  in  large  measure 
in  military  instructional  design  (practice, 
reinforcement,  knowledge  of  results,  small 
steps).  In  this  search  for  the  philosopher's 
stone  that  will  solve  all  problems,  McKeachie 
has  warned  that  "there  is  no  magic  way  to 
transmute  a  stubborn,  unmotivated  individual 
into  a  learner"  (McKeachie,  (16).  Not  only 
have  these  simplified  behaviorist  formulations 
been  adopted,  they  have  also  been  given  the 
status  of  eternal  truths  and  are  often  proce¬ 
dural  ized  in  handbooks.  One  recent  develop¬ 
ment  that  argues  against  the  advisability  of 
such  prescription  and  accompanying  proce¬ 
dural  guides,  is  the  demise  of  many  of  the 
so-called  laws  of  learning  that  were  once 
thought  to  be  immutable.  McKeachie  (1974)  has 
documented  the  fall  of  Thornkike's  laws  of 
effect  and  exercise  as  well  as  Skinner's 
primary  laws  of  practice,  knowledge  of  re¬ 
sults,  immediate  reinforcement,  and  gradual¬ 
ness.  Recent  research  has  indicated  that 
(a)  practice  need  not  be  overt,  (b)  knowledge 
of  results  does  not  have  to  be  immediate,  (c) 
reinforcement  should  be  informative  as  well 
as  inmediate,  and  (d)  learning  does  not  have 
to  take  place  in  small  steps  to  be  effective. 
While  recent  findings  have  not  destroyed 
some  of  the  basic  laws  of  learning,  they  have 
certainly  been  weakened  and  should  only  be 
applied  based  upon  consideration  of  each  unique 
situation. 

Furthermore,  much  of  the  research  on  teach¬ 
ing  methods  has  been  very  inconclusive.  Duben 
and  Taveggia  (3)  have  concluded  that  there  is 
no  difference  between  instructional  methods 
when  final  examinations  are  the  criterion. 
Danseraue  (4)  has  determined  that  there  are  no 
differences  in  instructional  effective- 
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ness  when  sequencing  of  instruction  is  varied. 
Aptitude-treatment  interactions  were  also 
found  to  be  insignificant  in  a  review  by 
Bracht{17).  In  general,  much  of  the  avail¬ 
able  research  evidence  indicates  that  there 
are,  perhaps,  more  exceptions  than  there  are 
rules  in  turn,  arguing  against  the  advisa¬ 
bility  of  procedural i zing  guides  for  the  design 
of  instruction  by  laymen.  To  use  the  analogy 
of  building  a  bridge,  the  engineer  must  have 
stable  rules  and  principles  to  apply,  much  as 
the  instructional  technologist  must  have 
stability  before  cookbooks  for  the  design  of 
instruction  can  be  applied  with  confidence. 


Prescribing  the  Learning  Environment 


The  debate  as  to  whether  instructional 
science  can  be  prescriptive  in  the  scientific 
sense  has  been  on-going  for  many  years  and 
has  intensified  lately.  At  issue  is  the  dis¬ 
tinction  posed  by  Simon (5)  between  arti¬ 
ficial  and  natural  sciences.  Is  it  possible 
to  obtain  scientific  understanding  of  man¬ 
made  institutions  such  as  educational  and 
training  systems?  Ebel  (6)  states  taht 
education  as  opposed  to  learning  is  not  a 
natural  phenomenon,  therefore,  we  do  not  need 
a  scientific  understanding  of  it;  we  need  to 
make  it  work.  A  similar  point  is  made  by 
Hardie  (7);  he  feels  that  we  cannot  ex¬ 
plain  and  predict  in  education  and  training; 
we  must  do  what  is  analagous  to  operations 
research  (scientific  preparation  for  deci¬ 
sions).  Cronback  (8)  also  favors  the 
decision  arrested  approach  for  instructional 
science  versus  the  conclusion  oriented  nature 
of  natural  science. 


Glass  (9)  has  asserted  that  too  little 
is  known  to  conceive  of  instructional  science 
as  a  prescriptive,  top-down  science.  His 
studies  of  average  effect  size  in  educational 
research  have  found  only  1/3  of  the  variance 
accounted  for  in  most  research  with  little  in 
the  way  of  patterns  emerging.  With  such  vari¬ 
ance  and  upredictabil ity,  how  can  policy  be  made 
by  the  scientist  or  engineer?  Glass's  con¬ 
clusion  was  that  there  were  too  many  unknown, 
immeasurable  and  uncontrollable  factors  to 
hope  that  a  prescriptive  science  could  emerge. 
The  warning  made  by  Judd  (10)  that  "there 
is  no  such  thing  as  pedagogical  formulas  - 
each  now  situation  is  unique,"  still  appears 
to  hold  true  today, 

8ased  upon  the  above  assessment,  what 
hope  is  there  for  a  science  of  instruction 
and  a  degree  of  structure  for  the  instruction¬ 
al  technologist?  Both  Cronbach  and  Glass 
have  called  for  decision  oriented  research, 
what  Glass  (9)  has  referred  to  as  general 
systems  theory.  In  this  approach,  the  amount 
of  prediction  possible  in  a  system  is  assessed, 
and  one  learns  to  cope  with  the  work  around 
unpredictabil ity. 

Another  issue  is  the  nature  of  the  sub¬ 
ject  matter  that  we  work  with.  Prescription 


directions  are  written  for  the  development  of 
explicit  behavioral  objectives,  for  drill  and 
practice,  extensive  feedback,  etc.  The  demand 
has  thus  been  to  seek  out  subject  matter  that 
can  be  reduced  to  easily  stated  behavioral 
objectives  that  can  be  delivered  via  programmed 
instruction  or  computers.  However,  all  know¬ 
ledge  that  we  may  want  to  convey  to  a  student 
is  not  explicit  and  may  not  fit  into  the  pro¬ 
grammed  instruction  format.  Polyani  (14)  has 
distinguished  between  two  forms  of  knowing: 
tacit  and  explicit;  the  tacit  form  being  more 
of  a  subsidiary  level  of  knowledge-that  which 
cannot  be  made  explicit.  This  distinction  is 
similar  to  that  discussed  by  Klein  (15) 
between  explicit  tasks  which  can  be  procedur¬ 
al  ized  in  the  form  of  guidebooks  and  those 
tasks  requiring  recognitional  capacity  that 
cannot  be  proceduralized. 

Technology,  as  it  is  currently  applied, 
and  procedural  guides  can  be  useful  for  learn¬ 
ing  environments  involving  learning  concrete 
subject  matter  for  which  behavioral  objectives 
can  be  specified;  however,  according  to  Broudy 
(2)  "the  more  distinctively  human  aspect 
of  experience  -  as  distinguished  from  the  robot 
aspect  -  seem  to  rely  heavily  on  the  tacit  forms 
of  knowing,  and  the  latter  will  not  take  place 
automatically  or  predictably  simply  as  the  con¬ 
sequence  of  learning  this  or  that  set  of  facts 
or  rules  or  acquiring  this  or  that  set  of 
skills.  The  military  instructional  technolo¬ 
gist,  while  not  a  cause  of  the  situation,  has 
gone  along  with  the  desires  of  the  military 
training  community  to  increase  the  explicit 
dimension  of  subject  matter  by  removing  all  in¬ 
formation  sources  that  are  not  considered  to 
be  essential  to  explicit  task  performance. 

Thus,  part  of  the  trap  we  may  find  ourselves  in 
involves  the  assumption  that  most  subject  mat¬ 
ter  can  be  delivered  by  the  mechanical  tutor 
approach  developed  by  Skinner.  While  such  an 
approach  may  be  suitable  for  superficial,  ex¬ 
plicit  drill  and  practice  this  form  of  learning, 
states  Broudy  (2).  is  "...  necessary  but  not 
a  sufficient  condition  for  the  more  global, 
evaluative,  and  cognitive  acts  of  the  pupil, 
and  indeed,  it  may  be  that  without  these  larger 
schemata  of  values,  goals,  and  commitments,  the 
explicit  learnings  will  be  harder  to  achieve 
and  render  functional." 


In  the  remainder  of  this  paper,  I  will  dis¬ 
cuss  how  the  adoption  of  a  balanced  cognitive 
perspective  may  aid  the  instructional  psycholo¬ 
gist  in  his  search  for  what  works. 


Emergence  of  Cognitive  Psycholog 


Many  of  the  problems  evident  in  the  behav- 
iorist  philosophy  of  instructional  design  and 
specification  can  be  traced  to  the  conviction 
that  the  instructional  materials  and  the  in¬ 
structor  are  responsible  for  learning,  not  the 
student.  Thus,  it  is  assumed,  if  every  detail 
in  the  external  learning  environment  is  speci¬ 
fied,  the  learning  will  take  place.  However, 
the  cognitive  perspective  places  much  greater 
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emphasis  on  the  student  and  his  internal  medi¬ 
ation  processes.  As  a  result,  each  learning 
event  is  somewhat  unique  and  cannot  be  pre¬ 
scribed  in  great  detail.  The  instructional 
technologist  must  then  focus  his  attention  on 
the  student,  his  motivation,  his  attention, 
his  strategies  for  processing  information; 
not  on  the  instructional  materials  and  deliv¬ 
ery  system  alone.  Once  the  instructional 
technologist  realizes  that  it  is  not  neces¬ 
sarily  the  materials  or  media  that  do  the 
teaching,  but  instead  the  internal,  cognitive 
mediation  of  the  student,  then  the  desire  to 
reduce  the  instructional  process  to  a  series 
of  procedural  steps  accompanied  by  infusions 
of  more  and  more  sophisticated  media  will  be 
lessened.  As  stated  by  Cronbach  (8),  ".  .  . 
technology  will  be  futile  in  education  until 
a  mere  basic  understanding  of  the  learning 
process  is  arrived  at." 

What,  then,  is  to  replace  the  structure 
that  was  provided  with  the  “automatic"  learn¬ 
ing  philosophy  consisting  of  practice,  imme¬ 
diate  feedback,  reinforcement,  and  overt 
action?  The  answer  lies  not  in  the  replace¬ 
ment  of  many  tried  and  true  techniques,  but 
more  so  in  knowing  when  and  where  to  apply 
such  techniques,  as  cognitive  elaboration, 
imagery,  analogies,  constructed  meaning,  and 
other  techniques  that  aid  students  in  dis¬ 
covering  and  creating  reality-based  learning, 
not  automatic  responding.  According  to 
Wittrock  (11),  the  learner  learns  when  he 
generates  perceptions  and  meaning  which  is 
consistent  with  prior  learning. 

The  cognitive  approach  places  responsi¬ 
bility  on  the  learner  to  generate  his  own 
meaning.,  for  instructional  materials  from  his 
unique  background,  attitudes,  experiences, 
etc.  According  to  this  view,  cognitive 
structures  must  be  formed  to  facilitate  learn¬ 
ing,  and  these  structures  can  most  readily  be 
understood  and  remembered  if  they  are  associ¬ 
ated  with  the  idiosyncrasies  of  the  learner. 

Future  Research  in  Instructional  Psychology 

Faced  with  the  lack  of  generality  of 
many  of  the  so-called  laws  of  learning  and 
the  apparent  failure  of  research  in  teach¬ 
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ABSTRACT 

Trends  in  simulation  training  are  reviewed,  highlighting  both  the  technological  advances 
and  the  recurrent  problem  areas  which  have  encouraged  and  plagued  instructional  system  devel¬ 
opers.  These  trends  are  expected  to  persist  into  the  80's.  In  general,  the  paper  draws  from 
the  simulation  literature  of  the  late  1970's.  It  clarifies  and  diagnoses  fundamental  human 
factors  issues  in  simulation  training.  Viewpoints  of  industry  and  military  researchers  are 
evaluated  and  integrated.  Recommendations  are  based  on  existing  data  which  support  concrete 
problem-solving  strategies.  Research  approaches  are  suggested  for  areas  where  knowledge  gaps 
exist.  The  following  topics  are  discussed:  (1)  Persisting  issues  in  motion,  fidelity,  and 
user  attitudes;  (2)  Interaction  of  training  device  instructional  features,  traditional  in¬ 
structor  functions,  and  clarity  of  behavioral  goals;  (3)  The  psychosocial  culture  of  simulator 
instructors;  (4)  Training  effectiveness  and  transfer:  The  problem  of  evaluation;  (5)  Simu¬ 


lator  sophistication  and  instructor  workload. 


INTRODUCTION 

A  characteristic  of  man  which  sets  him 
apart  from  the  lower  animals  is  his  extensive 
use  of  tools  to  interact  with  his  environment. 
Some  of  the  first  tools  were  weapons  for  defense. 
They  were  simple  to  understand  and  the  users 
were  easily  trained.  Through  the  ages,  man  has 
developed  increasingly  sophisticated  tools  which 
are  progressively  more  difficult  for  trainees  to 
comprehend.  The  instructor's  job  demands  have 
increased  in  a  direct  relationship  to  the  com¬ 
plexity  of  the  machine.  However,  the  ability 
of  the  trainee  to  comprehend  and  master  the 
machine  is  inversely-related  to  the  machine's 
complexity. 

Seventy-five  years  ago,  in  the  early  years 
of  aviation,  pilots  were  trained  on  the  job  to 
operate  their  flying  machines  (many  of  which 
were  not  as  complicated  as  today's  automobiles). 
However,  the  advent  of  today's  sophisticated 
airborne  weapon  systems,  coupled  with  diminishing 
fuel  supplies,  have  made  the  ground-based  aircrew 
training  device  a  necessity  and  a  new  set  of 
educational  and  training  parameters  has  surfaced. 
Modern  flight  simulators  have  introduced  a  new 
element  into  the  pilot  training  program  -  a 
computer  system  capable  of  reproducing  most  of 
the  dimensions  of  flight  in  a  safe,  adaptable, 
classroom  setting. 

The  simulator  set  into  motion  an  increase 
in  technological  creativity  which  rivaled  the 
industrial  revolution  in  terms  of  its  impact 
on  the  techniques  and  direction  of  education 
and  training.  With  the  advent  of  modern  trainers, 
another  characteristic  of  man  became  evident: 

His  idea  that  a  more  sophisticated  machine  would 
solve  his  problems.  The  approach  to  solving  the 
pilot  training  problem  has  been  focused  primari¬ 
ly  on  developing  more  complex  technology.  A 


standard  approach  to  pilot  training  with  simula¬ 
tors  has  not  been  developed,  and  as  Williges, 
Roscoe,  and  Williges  note,  the  most  common  mea¬ 
sure  of  training  effectiveness,  retention  of 
skills,  has  been  largely  ignored  in  the  evalua¬ 
tion  of  simulators  (1). 

The  addition  of  a  computer  -  assisted 
simulator  can  have  profound  implications  for 
training  beyond  the  obvious  need  to  deal  with 
complex  hardware  and  software.  The  psychological 
impact  on  an  instructor  pilot,  who  had  previously 
been  in  total  control  of  a  sophisticated  airborne 
weapon  system  (a  real-life,  hands-on,  natural 
classroom),  but  now  is  employing  a  sophisticated 
computerized  weapon  system  trainer  (WST)  to 
impart  skills  to  pilot  trainees,  is  one  such 
implication  worth  serious  consideration. 

The  modern  instructor  has  many  responsibili¬ 
ties  in  addition  to  maintaining  a  positive  rela¬ 
tionship  with  his  trainees.  His  efforts  are 
directed  at  the  orchestration  of  a  sophisticated 
multidimensional  learning  system.  Ideally,  with 
the  assistance  of  a  computerized  weapon  system 
trainer,  an  instructor  should  be  free  to  focus 
more  time  on  the  development  of  each  individual 
student.  Although  many  recent  documents  have 
assessed  the  mechanical  technology  available 
for  improving  aircrew  training  device  fidelity, 
there  seems  to  be  a  scarcity  of  research  directed 
toward  the  instructor's  needs.  What  are  the 
human  factors,  the  instructional  factors,  in 
modern  trainer  development? 

Many  of  the  problems  we  wish  to  discuss  in 
simulation  and  educational  technology  go  back  a 
long  way.  For  example,  Edwin  Link  encountered 
skepticism  and  roadblocks  in  user  acceptance 
with  his  first  instrument  flight  trainer,  the 
"Blue  Box".  This  device,  intended  for  general 
aviation  usage,  was  largely  ignored  by  that 


*  Staff  consultant.  Crew  Systems  Technology,  Boeing  Military  Airplane  Company. 
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community  and  found  its  early  acceptance  instead 
in  the  entertainment  field.  Whether  aviation's 
rejection  of  Link's  trainer  was  based  upon  its 
low  cost  or  its  humble  origins  in  the  piano  and 
organ  business,  remains  unknown. 

8y  1931  Link  had  become  a  wiser  marketer, 
and  was  able  to  triple  his  delivery  price  to 
$1,500.  He  also  taught  an  aviation-naive  Naval 
officer  to  fly  by  instruments  in  the  Blc0  Box, 
and  Navy  acceptance  was  quick  to  follow  (2). 

The  Blue  Box  was  not  a  flight  simulator  |>er  se 
but  a  cockpit  procedures  trainer  with  no  serious 
claim  to  realistic  aircraft  feel.  However,  it 
did  have  motion  and  was  a  trend-setter  in  that 
respect.  Fifty  years  later  we  remain  uncertain 
about  the  need  for  realistic  motion  in  flight 
simulation.  Modern  visual  systems  coupled  with 
certain  characteristics  of  human  psychophysiology 
make  it  possible  for  simulator  students  to  behave 
as  though  they  are  moving  without  actually 
moving. 

This  paper  is  an  outgrowth  of  a  routine 
literature  search  through  documents  dealing  with 
instructor/operator  console  input  systems.  It 
became  apparent  to  the  authors  that  certain 
fundamental  problems  in  training  technology 
remain  unsolved  or,  at  best,  glossed  over.  This 
condition  exists  despite  (perhaps  even  because 
of)  great  advances  in  training  device  sophisti¬ 
cation.  For  example,  simulator  builders  and 
buyers  place  great  emphasis  on  realism  and  physi¬ 
cal  fidelity;  yet  as  Fink  and  Shriver  point  out, 
the  factual  basis  for  this  approach  (and  its 
implications  for  transfer  of  training)  is  unclear 
(3).  On  the  same  point,  Adams  reminds  us  that 
a  simulator  is  a  teaching  machine,  not  an 
earthbound  aircraft  (4).  The  problem  of  simu¬ 
lator  fidelity  and  motion  is  discussed  below, 
as  is  the  related  issue  of  instructor/student 
attitudes  toward  simulation.  This  general 
domain  encompasses  situations  where  training 
device  sophistication  and  mandated  instructional 
features  may  have  outrun  the  supporting  empirical 
data. 

A  different  disjunction  between  supporting 
data  and  instructional  technology  is  apparent 
when  one  examines  the  training  and  responsibili¬ 
ties  of  instructors.  This  problem  represents  a 
case  in  which  a  problem  area  is  glossed  over  or 
perhaps  is  concealed  in  a  smokescreen  of  device 
sophistication.  The  reader  is  likely  aware  that 
the  training  of  instructors  for  their  role  is 
largely  unstandardized,  and  perhaps  even  haphaz¬ 
ard  in  some  settings.  We  will  discuss  five  issues 
in  training  technology,  highlighting  current 
practices,  and  the  likely  myths  related  to  each 
issue.  The  issues  are  (a)  Motion,  Fidelity,  and 
User  Attitudes;  (b)  Interaction  of  Training 
Device  Instructional  Features,  Instructor 
Functions,  and  Clarity  of  Behavioral  Goals; 

(c)  The  Psychosocial  Culture  of  Instructor/ 
Operators;  (d)  Training  Effectiveness,  Transfer 
and  Evaluation;  and  (e)  Simulator  Sophistication 
and  Instructor  Workload.  Although  our  approach 
draws  mainly  from  the  documents  of  other 
researchers  in  the  instructional  technology 


field,  we  hope  that  our  psychological/educational 
emphasis  will  stimulate  new  directions  in  think¬ 
ing  about  these  old  problems. 


PERSISTING  ISSUES  IN  SIMULATOR  MOTION, 
FIDELITY,  AND  USER  ATTITUDES 

That  motion  systems  contribute  substantial¬ 
ly  to  flight  simulator  costs  is  well  known.  But 
as  Caro  has  argued,  we  can  with  confidence 
neither  eliminate  motion  from  simulation  nor 
insist  upon  its  inclusion  in  the  interests  of 
simulation  training  effectiveness  and  efficiency 
(5).  Caro  cites  the  study  of  Jacobs  and  Roscoe 
in  which  beginning  flight  trainees  failed  to 
show  a  transfer  advantage  from  washout  motion 
compared  with  no-motion  control  subjects  (6). 
Citing  five  other  studies  on  the  comparative 
benefits  of  simulation  motion,  Caro  suggests 
that  motion  may  improve  the  efficiency  of  train¬ 
ing  if  not  the  effectiveness,  and  that  a  dynamic 
flight  simulator  may  give  the  trainee  a  slight 
edge  in  control  strategies  arising  from  inflight 
emergency  maneuvers.  [An  analogy  suggests  it¬ 
self:  Investing  in  simulator  motion  systems 
,„uy  be  like  buying  and  saving  new-issue  postage 
stamps.  At  worst,  the  investor  retains  a  useful 
commodity  (though  subject  to  inflation  shrinkage) 
but  enjoys  no  capital  gain.  In  the  best  case, 
some  stamps  may  return  a  large  future  dividend, 
provided  the  bearer  is  at  the  right  place  at  the 
favorable  moment.] 

Once  a  decision  in  favor  of  motion  is 
reached,  the  question  of  required  degree  of 
motion  realism  (aircraft  "feel"  and  handling 
qualities)  comes  to  the  fore.  The  Link  was  an 
effective  trainer  whose  motions  were  but  crude 
caricatures  of  real  airplane  responses.  Simu¬ 
lator  "feel"  is  an  area  where,  from  design  to 
eventual  evaluation,  pilot  opinions  are  the 
dominating  criteria.  Yet,  as  Caro  points  out, 
training  effectiveness  seems  not  to  follow  from 
motion  realism  in  any  orderly  manner  (5).  Adams 
criticizes  this  dependence  on  pilot  ratings  for 
evaluating  flight  simulators  along  broader  lines 
than  motion  considerations  jpjr  se  (4).  Adams 
questions  whether  high  physical  fidelity  inevi¬ 
tably  leads  to  high  positive  transfer.  He  notes 
that  the  transfer/similarity  relationship  can 
be  decoupled,  in  that  mockups  have  produced  as 
much  positive  transfer  as  high  fidelity  simula¬ 
tors,  and  that  transfer  has  been  recorded  in 
situations  manipulated  to  actually  reduce 
physical  fidelity  (4).  A  similar  argument, 
advanced  by  Fink  and  Shriver,  was  that  simpli¬ 
fied  training  devices  are  adequate  for  bringing 
military  maintenance  trainees  to  the  apprentice 
level.  High  skills  development  can  be  left  to 
the  on- job  environment  (3). 

An  excellent  brief  summary  of  literature 
bearing  on  the  fidelity/transfer  relationship 
was  prepared  by  R.  L.  Miller,  who  also  derived 
several  models  of  the  general  relationship 
between  simulator  cost,  effectiveness,  and 
degree  of  physical  fidelity  (the  original  model 
coincidentally,  was  prepared  by  R.  B.  Miller, 
also  at  Wright  Patterson  AFB,  in  1954)  (2). 


We  turn  now  to  the  role  of  user  attitudes 
(or  acceptance)  on  training  device  effective¬ 
ness.  Hopkins  has  said  that 

Almost  any  kind  of  simulator  can  be 
used  effectively  in  a  well-designed 
training  program  with  specially- 
trained  instructors  and  highly  moti¬ 
vated  instructors  and  students  (7). 

This  optimistic  statement  implies  that  we  could 
get  by  with  less  in  the  way  of  motion  systems 
and  physical  fidelity  and  still  train  effective¬ 
ly  if  we  had  well-trained  and  enthusiastic 
instructors  and  highly-motivated  students  with 
good  imaginations.  The  problems  in  instructor 
preparation  will  be  discussed  later,  and  we 
will  confine  our  remarks  here  to  user  motivation 
(in  the  present  context,  we  assume  that  "moti¬ 
vation"  and  "morale"  mean  similar  things,  and 
that  the  word  "attitude"  is  a  close  relative 
of  the  others.) 

Anyone  who  has  slogged  through  the  shifting 
sands  of  motivation  research  would  likely  agree 
that  unless  the  human  performer  is  internally 
motivated,  external  incentives  are  not  going 
to  sustain  task  commitment  for  very  long.  The 
trainee  must  want  to  be  trained  and  to  perform 
the  task  subsequently.  The  motivational  demands 
on  the  instructor  are  equally  significant.  We 
shall  later  address  the  motive/attitude  complex 
of  instructors  and  will  only  point  out  here 
another  "axiom":  Students  readily  pick  up  the 
attitudes  of  their  instructors  about  simulator 
fidelity,  usefulness,  and  general  worth.  In 
short,  the  acceptability  of  a  training  device 
appears  to  be  in  part  a  function  of  students' 
attitudes  about  that  device,  and  students' 
attitudes  are  in  turn  partly  (if  not  wholly) 
a  functon  of  their  instructors'  attitudes 
about  the  device. 

We  generally  believe  that  pilot  acceptance 
of  simulators  increases  with  degree  and  fidelity 
of  simulation,  and  that  reduced  fidelity  is 
likely  to  engender  negative  student  pilot 
attitudes.  Hopkins  charges  that  this  belief 
is  "...largely  a  myth  that  has  been  fostered 
and  perpetuated  by  those  interested  in  develop¬ 
ment  of  more  complex  and  more  costly  simulators. 
There  is  no  evidence  to  suggest  that  pilot 
acceptance  of  simulators  is  a  significant 
problem"  (8).  Hopkins  then  makes  one  of  the 
more  novel  (and  concrete)  suggestions  for 
assuring  pilot  acceptance  to  be  found  in  the 
training  literature: 

For  the  beginning  student  pilot, 
acceptance  of  considerably  less  than 
perfect  fidelity  in  the  simulator 
should  be  possible  if  flight  in  the 
aircraft  is  made  contingent  upon 
achievement  of  specified  levels  of 
performance  in  the  simulator 
(p.  538). 

The  psychologist  recognizes  this  strategy  as 
the  "Premack  Principle,"  which  involves  using 
a  highly  preferred  behavior  to  reward  prior 
performance  of  a  less-preferred  behavior  (9). 


Would  it  be  irresponsible  to  hypothesize 
that  strong  positive  student  and  instructor 
attitudes  (and  high  task-oriented  motivation  on 
the  parts  of  both)  would  not  only  compensate  for 
lowered  physical  fidelity,  but  perhaps  even 
reduce  simulator  motion  system  requirements? 

Such  a  hypothesis  may  have  some  appeal  from  a 
cost  viewpoint,  but  Alsobrook's  data  suggests 
the  hypothesis  is  in  trouble  (10).  Alsobrook 
surveyed  air  force  pilots  and  navigators  as  well 
as  top-level  managers  from  six  major  air  force 
commands.  Strong  hints  of  negative  attitudes 
toward  simulation  in  lieu  of  flying  hours  were 
manifested  by  both  aircrews  and  commanders. 
However,  attitudes  did  tend  to  become  more  posi¬ 
tive  as  training  devices  increased  in  instruc¬ 
tional  adequacy,  and  in  fidelity.  The  negative 
attitudes  were  pertinent  to  simulator  quality, 
obsolescence,  and  to  the  difficulties  of  simu¬ 
lating  the  flying  characteristics  of  fighter 
aircraft  (with  instructor  skill  and  dedication 
receiving  generally  good  marks).  Taken  together, 
Alsobrook's  data  imply  that  we  fall  short  in  the 
area  of  cultivating  strong  positive  attitudes 
toward  simulation. 


INTERACTION  OF  TRAINING  DEVICE  INSTRUCTIONAL 
FEATURES,  INSTRUCTOR  FUNCTIONS, 

AND  CLARITY  OF  BEHAVIORAL  GOALS 

Even  now,  there  is  substantial  applied 
research  evidence  that  much  of  the 
training  conducted  in  expensive  simu¬ 
lators  could  be  accomplished  in  less- 
expensive  devices  if  the  training 
programs  used  with'  them  were  properly 
designed  and  conducted...  a  realistic 
simulator  is  a  poor  substitute  for 
competent  training  (11). 

Although  it  is  currently  fashionable  to  attack 
the  school  of  psychology  known  as  behaviorism 
for  its  disinterest  in  (or  denial  of)  mental 
processes  and  physiological  factors  in  learning 
and  motivation,  in  fairness  to  Skinner  and 
others  we  must  recognize  behaviorism's  enduring 
contributions.  These  include  revolutionary 
psychiatric  treatment  models,  strategies  for 
managing  training  with  the  retarded,  the  delin¬ 
quent,  and  the  culturally  deprived,  techniques 
for  modifying  nonproductive  behaviors  of 
industrial  workers,  and  programmed  instruction. 

The  breadth  and  subtlety  of  behaviorism's 
influence  is  evident  in  our  unquestioning 
acceptance  of  the  value  of  positive  feedback/ 
reinforcement  in  training  management,  and  in  our 
strong  emphasis  on  establishing  clear  behavioral 
goals  in  instructional  system  development.  The 
latter  is  perhaps  behaviorism's  most  useful 
legacy  for  skills  training  settings:  Know 
exactly  what  behaviors  your  trainees  must  acquire; 
decide  exactly  the  level  of  performance  you  will 
accept;  and  pretest  your  trainees  to  determine 
the  magnitude  of  the  training  task.  A  bumper 
sticker  for  Instructors  might  read  "Where  are 
we  now?  Where  are  we  going?  How  will  we  know 
when  we  get  there?" 
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The  importance  of  having  concrete  behavioral 
goals  is  fairly  obvious  when  we  consider  teaching 
young  people  to  fly  modern  aircraft.  Instructors 
will  also  benefit  from  having  clear  behavioral 
goals.  This  is  true  both  in  terms  of  their  own 
training  as  instructors,  and  in  terms  of  their 
teaching  relationship  with  their  students  (What 
do  I  do?  How  exactly  do  I  do  it?  When  do  I  do 
it?)  The  literature  suggests,  however,  that  in 
training  instructors  we  do  our  poorest  job  of 
providing  behavioral  guidelines.  Instructor 
training  is  nonstandard  at  best,  and  extensive 
practice  in  operating  the  training  device  to 
fully  exploit  its  features  is  acquired  on-the- 
job  (maybe) ! 

On  the  full  exploitation  of  simulator  in¬ 
structional  features,  Blaiwes,  Puig,  and  Regan 
have  called  attention  to  the  occasional  communi¬ 
cation  gap  between  the  simulator  developer/ 
builder  and  the  system  user  (12).  This  gap, 
which  develops  shortly  after  delivery  and  in¬ 
stallation,  serves  to  keep  the  user  ignorant  of 
some  of  the  device's  instructional  features, 
thereby  blocking  full  exploitation  of  training 
capability.  Caro  refers  to  this  state  of  affairs 
as  one  where  the  simulator  is  "dropped  over  the 
fence  at  night"  (13).  The  implication  is  that 
after  delivering  the  training  device,  the  manu¬ 
facturer  fails  to  provide  adequate  follow-on 
support. 

The  literature  also  reflects  a  growing 
interest  in  syllabus  preparation  where  simulator 
instructional  features  are  emphasized,  in  con¬ 
trast  to  earlier  approaches  in  which  the  syllabus 
was  more  closely  bound  to  a  real  aircraft. 

Related  steps  in  specifying  instructor  behaviors 
are  exemplified  by  Charles'  functional  analysis 
and  breakdown  of  IP  duties  (14);  and  by 
Montemerlo's  discussion  of  current  simulator 
instructional  features  (15). 

Although  Montemerlo's  list  of  instructional 
features  points  in  the  direction  of  concrete 
instructor  responses,  his  purpose  was  to  explore 
instructional  features  which  could  be  automated, 
thus  freeing  the  instructor  for  a  more  managerial 
role.  This  suggestion  is  in  line  with  the  notion 
of  many  in  the  (military)  training  community  that 
instructors  should  be  eliminated  altogether  (thus 
eliminating  the  thorny  problems  of  instructor/ 
operator  station  design  and  instructor/operator 
jurisdictional  disputes).  However,  we  see 
totally  automated  training  as  still  over  the 
horizon  in  aviation.  Certainly  the  beneficial 
effects  of  automating  certain  aspects  of  simula¬ 
tion  training  seem  self-evident,  particularly  in 
the  area  of  student  performance  analysis.  Con¬ 
sider  a  simulator  with  the  capability  to  diagnose 
subtle  student  errors,  or  recognize  maneuvers 
needing  remedial  practice,  all  without  specific 
input  from  the  instructor.  This  "smart  grader" 
or  teaching  assistant  would  then  automatically 
switch  into  a  part-task  trainer  mode,  repeating 
problem  segments,  reassessing  student  response 
providing  slow-motion  and  playback  as  required, 
and  perhaps  presenting  a  visual  or  synthetic 
vocal  text  in  support  of  its  tutoring.  The  human 
instructor  can  do  all  this,  but  there  is  no 


assurance  that  it  is  done  uniformly  well.  The 
instructor  could  use  this  new  freedom  to  observe 
and  assess  the  larger  picture  of  the  student's 
response  to  the  general  learning  environment 
( i . e. ,  he  can  make  value  judgments). 

Perhaps  the  push  for  maximum  fidelity  in 
flight  simulation  has  dominated  current  research 
thought  to  the  point  where  the  features  which  can 
best  be  utilized  through  computerized  simulation 
training  have  been  overlooked.  What  technology 
exists  now  to  aid  the  instructor  in  evaluation 
and  analysis  of  trainee  performance?  How  are 
instructor  pilots  trained  to  utilize  these 
unique  simulator  capabilities?  It  may  well  be 
that  instead  of  being  scornful  of  the  synthetic 
flight  trainer's  lack  of  fidelity  and  "feel"  we 
should  consider  its  full  feature  potential  and 
the  unique  training  capabilities  it  offers  the 
instructor. 

The  challenge  is  to  achieve  an  appropriate 
menu  of  training  device  features  to  support 
instructors  in  their  teaching  functions  so  that 
students  can  attain  the  training  goals  set  for 
them.  At  present,  we  seem  technically  better 
at  specifying  training  goals  and  instructor 
activities  than  we  are  at  isolating  the  optimal 
array  of  device  features. 

This  latter  issue  received  emphasis  in  a 
paper  by  Cream,  Eggemeier,  and  Klein,  whose 
criticality-frequency-difficulty  (C-F-D)  analysis 
assists  the  training  device  user-builder  design 
team  to  determine  mandatory  instructional  fea¬ 
tures  and  exclude  less-cost-effective  features 
(16).  A  summary  of  the  C-F-D  decision  model 
cannot  do  it  justice  here,  but  in  essence  the 
model  says,  "If  a  feature  is  not  critical  in 
meeting  training  goals,  if  the  feature  repre¬ 
sents  a  situation  rarely  encountered  in  the  real 
environment,  and  if  the  feature  is  technically 
difficult  to  achieve,  possibly  irrelevant,  and 
certainly  costly,  then  exclude  it." 

The  key  to  applying  the  C-F-D  analysis  is 
the  clear  specification  of  training  goals  in 
student  behavior  terms.  On  this  latter  point, 
Williges,  Roscoe,  and  Willi ges  have  asked  a 
provocative  question  bearing  on  the  matter  of 
behavioral  goals:  What  is  ideal  pilot  behavior 
and  what  is  the  standard  for  recognizing  correct 
performance  (1)?  Do  the  appropriate  concrete 
behaviors  involve  staying  in  tolerance,  perform¬ 
ing  within  the  norms,  or  the  judgments  of  ex¬ 
perts?  Expressing  training  goals  in  concrete 
behavioral  terms  may  be  more  easily  said  than 
done. 


TRAINING  EFFECTIVENESS,  TRANSFER, 

AND  EVALUATION 

Perhaps  the  thorniest  problem  of  all  is  the 
evaluation  problem:  It  impacts  the  student,  the 
instructor,  and  the  device  itself.  We  have  sub¬ 
scribed  to  the  positive  transfer  of  training 
model,  that  behaviors  learned  in  one  setting 
will  carry  over  to  a  different  setting  to  the 
extent  that  stimulus  and  response  elements  in 
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the  two  settings  are  similar.  In  general,  that 
model  has  served  us  well.  It  is  the  basis  for 
the  guest  for  psycho/physical  fidelity  and,  as 
the  rationale  for  realism  in  military  training, 
was  given  good  marks  by  World  War  II  GIs  for  its 
preparation  for  combat.  As  Adams  has  reminded 
us,  positive  transfer  is  what  we're  about,  and 
positive  transfer  tends  to  grow  out  of  fidelity 
(the  key  concept  being  similarity)  (4). 

After  reminding  us  of  the  paucity  of  solid 
data  on  positive  transfer  in  aviation  training, 
Adams  points  out  that  the  actual  level  of  trans¬ 
fer  obtained  has  been  of  the  order  of  10-25%, 
and  he  offers  sharp  criticism  of  the  transfer 
paradigm  as  applied  to  aviation  training.  His 
most  provocative  point  is  that  the  outcome  of 
the  test  for  positive  transfer  is  (must  be)  a 
foregone  conclusion,  in  that 

The  subjects  of  an  experimental  group 
must  fly  the  aircraft  well  on  the 
first  flight  after  simulation  train¬ 
ing.  If  not,  they  would  not  be  allowed 
to  fly,  and  so  there  would  be  no  mea¬ 
sures  and  no  experiment  (p.  713). 

Adams  offers  instead  an  alternative  rating  plan 
which  emphasizes  (1)  whether  the  prototype 
simulator  meets  the  provisions  of  the  handful 
of  fundamental  and  reliable  laws  of  learning 
psychology,  and  (2)  whether  the  device  is  suffi¬ 
ciently  like  other  similar  devices  which  we 
believe  to  be  effective  trainers.  It  is  easy 
to  share  Adams'  discouragement  with  the  training 
transfer  paradigm  when  one  considers  comments 
such  as  Val verde's  that  the  contradictory  re¬ 
sults  which  characterize  the  training  transfer 
studies  might  be  traced  to  confounding  or  unas¬ 
sessed  variables  in  the  criterion  measures,  the 
student  group  assignment  procedures,  the  in¬ 
structor,  and  the  instructional  sequence  (17). 

Adams  takes  critical  aim  at  another  prac¬ 
tice  common  in  training  device  evaluation,  the 
reliance  on  pilots'  ratings  in  judging  the 
degree  to  which  the  simulator  accurately  models 
the  (aircraft)  system  (4).  He  grants  the  abili¬ 
ty  of  pilots  to  assess  a  flight  simulator  for 
its  physical  fidelity,  but  questions  whether 
pilots  are  trained  to  detect  crucial  and  subtle 
elements  of  psychological  fidelity.  He  reviews 
data  showing  that  simulator  ratings  change  as 
both  pilots  and  simulator  students  gain  expe¬ 
rience;  and  he  discusses  data  that  challenge 
the  assumed  relationship  between  fidelity  and 
student  performance.  Finally,  Adams  attacks  the 
premise  Implicit  in  the  current  rating  method 
that  a  flight  simulator  is  a  ground-based  air¬ 
craft,  reminding  us  instead  that  a  flight  simu¬ 
lator  is  a  teaching  machine  and  that  it  should 
be  rated  as  such.  Clearly,  this  paper  is 
important  reading. 


THE  PSYCHOSOCIAL  CULTURE  OF  INSTRUCTORS 

It  Is  clear  that  teachers  will  retain  their 
crucial  role  in  instructional  systems  in  the 
foreseeable  future.  It  is  also  clear  that 


instructor  recruitment,  preparation,  attitudes, 
work  schedules,  and  reward  systems  are  current¬ 
ly  subject  to  great  variance.  These  aspects 
illustrate  what  we  mean  by  the  term  “psycho¬ 
social  culture";  and,  although  there  is  little 
hard  evidence  that  social  and  psychological 
aspects  of  instructors  either  enhance  or  degrade 
training  effectiveness,  common  sense  tells  us 
that  there  are  good  and  poor  instructors,  good 
and  poor  training  programs,  and  many  interme¬ 
diates  on  both  counts. 

Caro,  Shelnutt,  and  Spears  discuss  the 
characteristics  of  more  effective  and  less 
effective  training  units  and  provide  extensive 
observations  on  the  qualities  and  behaviors  of 
instructor  personnel  associated  with  both  (13). 
From  their  observations  emerges  the  following 
profile  of  instructors  in  effective  units:  The 
instructor  was  an  above-average  aircrew  member 
who  sought  the  training  assignment  to  diversify 
his  career;  he  is  a  mature  and  stable  person 
whose  final  selection  was  made  by  his  eventual 
instructor  peers;  and  his  rapport  with  both 
students  and  peers  was  good.  His  instructional 
duties  were  given  top  priority  in  the  training 
unit,  and  he  enjoyed  flexibility  in  scheduling 
his  secondary  duties  so  that  conflict  among 
his  assignments  was  minimal. 

As  an  instructor-in-training,  this  new 
member  of  an  effective  training  unit  first 
trained  as  a  student,  getting  the  full  course 
in  both  the  aircrew  training  device  and  subject 
aircraft.  Then  he  observed  the  full  training 
of  two  students  under  direction  of  a  senior 
instructor  who  was  also  his  tutor  on  device 
capability  and  operation.  Next,  the  instructor 
trainee  gave  segments  of  training  to  two  senior 
instructors  playing  the  student  role  (perfor¬ 
mance  reviews  and  critiques  were  part  of  this 
phase  of  instructor  training).  Finally,  the 
instructor  trainee  conducted  supervised  train¬ 
ing  with  two  "real"  students,  an  activity 
analagous  to  "practice  teaching." 

If  we  are  to  support  the  crucial  role 
played  by  the  instructor  in  simulation  settings, 
our  near-term  strategy  ought  to  be  to  take  steps 
to  enhance  the  instructor  "culture.”  Those 
steps  should  include  standardizing  and  formaliz¬ 
ing  the  recruitment  of  instructor  personnel 
along  lines  described  by  Caro  et  al^,  structuring 
and  formalizing  the  training  o?~instructors 
(again,  see  Caro  et  al),  assuring  that  the 
training  assignment  Has  the  highest  priority 
among  the  officer's  other  duties,  and  generating 
a  proper  and  uniform  reward  structure  both 
during  the  instructional  tour  of  duty  and  in  the 
subsequent  assignment  of  Instructors  (13).  By 
no  means  should  we  tolerate  situations  where 
instructors  and  training  managers  voice  clearly 
negative  attitudes  about  the  worth  of  training 
devices  in  general,  or  where  a  conscientious 
instructor  actually  loses  ground  in  the  person¬ 
nel  rating  arena  relative  to  his  non-instructing 
peers. 

Along  those  same  lines,  we  need  to  assure 
that  proficiency  ratings  of  instructors,  already 


175 


an  elite  group,  are  not  depressed  by  adherence 
to  a  forced-choice  rating  system  not  designed 
for  upper-level  groups.  Caro  is  explicit  on 
the  role  played  by  instructors  and  other  sup¬ 
portive  elements  in  instructional  systems  (11) 

It  is  becoming  evident. . .that  the 
training  vehicles-the  simulators 
principally,  but  also  the  training 
aircraft-are  less  important  in  many 
respects  than  are  the  instructors 
and  the  organization  and  content 
of  the  training  programs  (p.  504). 


SIMULATOR  SOPHISTICATION  AND 
INSTRUCTOR  WORKLOAD 

The  aircrew  training  device  instructor's 
workload  includes  the  following:  Preparation 
of  student  data,  syllabus  development,  operation 
of  instructor  console,  evaluation,  brief  and 
debrief,  training  of  other  instructor  pilots, 
data  management,  and  noninstructional  duties. 
This  listing  does  not  contain  all  the  workload 
responsibilities  of  an  instructor  but  it  helps 
to  illustrate  that  he  must  wear  three  hats 
simultaneously:  Pilot,  educator,  and  computer 
operator.  Most  reports  on  simulation  training 
concentrate  on  the  instructor's  role  as  a  pilot 
and  computer  operator.  The  instructor  workload 
section  of  this  report  will  focus  on  the 
instructor's  role  as  educator. 

Many  engineers,  pilots,  and  design  personnel 
tend  to  view  the  training  simulator  as  a  ground- 
based  aircraft  rather  than  a  teaching  machine. 
This  attitude  focuses  attention  on  how  to  devel¬ 
op  greater  physical  fidelity,  and  the  standardi¬ 
zation  of  the  computer  input  devices  at  the 
instructor/operator  station  to  provide  better 
simulator  control.  Perhaps  some  insights  into 
the  educational  aspects  of  training  would  help 
in  designing  simulation  technology  which  would 
enhance  the  instructor's  role  as  educator  as 
well  as  pilot  and  computer  operator. 

The  instructor's  educational  workload  has 
many  facets,  including  course  content,  course 
presentation,  trainee  performance  analysis,  and 
maintenance  of  a  constructive  relationship  with 
his  trainees.  An  instructor's  conception  of  a 
trainee  has  an  effect  on  this  relationship. 
Figure  1,  Trainee  Cognate  Areas,  illustrate 
aspects  of  a  trainee  with  which  an  instructor 
must  interact. 


Psychomotor 

-  Eye-hand  coordination 

-  Control  inputs 

-  Movement  sequence 

-  Emergency  response 

-  Control  "feel" 

-  Reflexes 


Cognitive 

-  Decision  making 

-  Prediction 

-  Memory 

-  Judgment 

-  Evaluation 

Affective 

-  Attitudes 

-  Morale 

-  Self-Image 


FIGURE  1:  TRAINEE  COGNATE  AREAS 


This  diagram  illustrates  some  of  the  human 
elements  which  have  an  effect  on  learning 
quality,  quantity,  and  rate.  Educationally, 
the  instructor’s  workload  involves  interaction 
with  all  three  of  the  cognate  areas:  psycho¬ 
motor,  cognitive,  and  affective.  The  training 
which  an  instructor  receives  to  prepare  him  for 
interaction  with  these  three  trainee  areas  is 
not  adequate  in  most  programs,  especially  in 
the  affective  area.  Charles  reviewed  simulation 
training  operations  and  concluded  that  instructor 
consoles  are  not  designed  for  training  implemen¬ 
tation  and  that  the  instructor  pilot  i  trained 
in  neither  simulator  utilization  nor  i  i  how  to 
instruct  (14).  For  example,  Caro  et  1  indicate 
that  instructor  training  covers  a  bro.  i  continu¬ 
um  ranging  from  brief  "checkout"  on  the  aircrew 
training  device  instructor  console,  to  a  compre¬ 
hensive  and  systematic  training  program  (13). 
Thus,  the  preparation  provided  to  simulation 
instructors  is  inconsistent  at  best,  and  recogni¬ 
tion  of  the  need  for  a  standardized  training 
approach  is  one  of  the  first  steps  toward  defin¬ 
ing  the  training  program  for  instructors. 
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The  workload  of  competent  instructors  in¬ 
volves  a  repertoire  of  behaviors  much  broader 
and  more  intricate  than  the  simulator  syllabus 
guidelines  would  indicate.  Just  as  the  pro¬ 
fessional  flyer  has  a  much  greater  understanding 
of  flight  than  could  be  gained  from  reading 
through  a  flight  procedures  manual,  the  pro¬ 
fessional  instructor  has  more  understanding  and 
control  of  instructional  methods,  media,  and 
systems  than  a  course  syllabus  would  supply. 

Many  opinions  presently  exist  on  the  issue 
of  who  should  instruct  and  what  the  workload 
should  include.  Perhaps  the  answer  to  the 
following  question  would  help  in  solving  the 
issue:  Is  the  instructor  a  dedicated  pilot 
assigned  to  a  tour  of  teaching  in  a  ground- 
based  aircraft  or  is  the  instructor  a  dedicated 
educator  at  the  controls  of  a  sophisticated  air 
training  device? 

It  appears  that  instructor  workloads  have 
increased  along  with  achievements  in  simulator 
sophistication,  to  the  point  that  instructors 
run  the  risk  of  being  unable  to  function  as 
teachers  because  of  the  demands  of  console 
operation.  Moreover,  some  weapons  system  train¬ 
ers  (WST)  require  several  instructors  working 
in  concert.  Bolwerk  describes  the  actions  of 
an  S-3A  instructor  crew  (device  2F-92)  and  uses 
the  phrase  "...Mack  Sennett  comedy..."  to 
describe  their  antics  (18).  He  also  recounts 
a  case  where  two  stand-alone  devices  were 
integrated  into  a  WST  which,  it  was  thought, 
would  reduce  the  number  of  instructors.  Con¬ 
trary  to  expectation,  the  WST  required  the  full 
complement  of  instructor  personnel  from  both 
stand-alone  devices. 

Reliability  problems  also  became  more 
frequent  with  increasing  sophistication,  and 
reliability  problems  produce  down-time  which, 
in  turn,  produces  scheduling  difficulties  (7). 

As  scheduling  difficulties  arise,  there  can 
develop  a  temptation  to  move  toward  a  square¬ 
filling  utilization  approach  to  training.  For 
example,  if  a  syllabus  calls  for  X  number  of 
fault  insertions,  and  run  time  has  been  fore¬ 
shortened  by  simulator  down-time,  the  result 
can  be  a  rapid-fire  program  of  fault  insertion 
with  student  management  of  those  faults  pre¬ 
ordained  to  be  inadequate:  Batching  leads  to 
botching,  so  to  speak;  and  student  attitudes 
about  simulation  training  can  suffer  as  a  re¬ 
sult.  Sometimes  high  sophistication  leads  to 
information  overload  on  the  part  of  the  instruc¬ 
tor.  Information  overload  can  result  from 
building  more  sophistication  into  the  device 
than  is  actually  needed.  Cream  et  aX  describe 
one  simulator  designed  to  provide  hard  copy 
of  student  performance  parameters,  updated  15 
times  a  second  (16).  The  resulting  mound  of 
printout  paper  was  neither  anticipated  nor 
desired  by  the  user. 

A  comnon  proposal  these  days  is  to  auto¬ 
mate  many  of  the  manual  tasks  of  the  instructor, 
particularly  in  reference  to  console  operation. 
The  goal  is  to  free  the  instructor  to  make  value 
judgments  about  student  progress,  much  as  an 


automatic  pilot  frees  an  aircraft  commander 
from  routine  tasks.  Automation  would  make  the 
instructor  the  manager  of  training  rather  than 
the  operator  of  the  simulator.  This  is  a  noble 
goal,  and  most  would  agree  that  we  ought  to  try 
to  achieve  it.  At  the  same  time,  we  need  to 
anticipate  a  likely  tradeoff  here.  It  is  pos¬ 
sible  that  console  function  automation,  in 
reducing  the  motor/manual  workload  of  instruc¬ 
tors,  might  simply  be  traded  for  a  greatly 
increased  cognitive  workload  where  the  instruc¬ 
tor  now  has  to  scan  more  displays,  integrate 
more  information,  and  make  more  decisions, 
though  his  hands  rest  on  the  console  work 
surface.  In  that  sense,  the  job  of  instructor 
might  take  on  some  of  the  stressful  and  unde¬ 
sirable  aspects  of  the  job  of  an  air  traffic 
controller  in  a  dense  traffic  environment. 

Even  in  the  best  case,  where  instructor 
workload  is  pared  to  a  minimum  by  an  intelli¬ 
gent  automated  computer,  there  are  risks.  Not 
only  do  we  widen  the  instructor-student  inter¬ 
personal  distance,  but  moreover,  the  instructor 
is  left  standing  in  the  shadow  of  the  computer. 
These  possibilities  call  for  some  thoughtful 
and  careful  research  in  instructor  task  analysis 
and  function  allocation. 

An  opposite  sort  of  outcome  of  console 
automation  is  that  automated  training  devices 
might  give  the  instructor  too  little  to  do. 

Our  guess  is  that  this  unlikely  circumstance 
would  stimulate  instructors  to  become  creative 
and  devise  some  effective  off-line  instructional 
follow-on  strategies  which,  according  to  Bolwerk, 
continue  to  elude  the  training  community  (18). 

A  technique  which  holds  promise  for  both 
alleviating  instructor  workload  and  as  a  basis 
for  an  instructional  follow-on  strategy  is  that 
of  peer  training  as  described  by  Caro  (11): 

Trainees  themselves  are  used  to  assist 
fellow  trainees  in  many  simulator 
t-'ining  activities.  This  technique 
has  been  found  particularly  useful 
with  respect  to  cognitive  problem¬ 
solving  activities  such  as  those 
which  occur  during  navigation  prob¬ 
lems.  Simulators  are  particularly 
well -suited  for  peer  instruction 
because  the  instructor  can  be  remove! 
from  the  cockpit  area  without  creating 
flight  safety  problems  with  relatively 
unskilled  trainees  (p.  506). 

The  issue  is  not  whether  we  shall  find 
simulator  training  acceptable,  because  our  only 
alternative  is  to  maximize  the  present  state  of 
the  art.  The  resistance  we  are  seeing  is 
reminiscent  of  the  reactions  of  the  old  horse 
cavalrymen  to  the  advent  of  mechanized  fighting 
vehicles,  and  to  the  broader  resistance  of  the 
military  establishment  to  airpower  in  the  early 
days.  Our  current  confusions  about  the  role  of 
the  eventual  user  in  simulator  design,  about 
the  essential  instructional  features  to  be 
included,  about  developing  simulator-based 
syllabi,  about  the  number  and  pacing  of  fault 
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Insertions,  about  the  roles  of  instructor  and 
technician,  about  optimum  usage  of  the  device, 
and  about  student,  instructor,  and  device  evalu¬ 
ation  will  all  pass  in  due  time. 


CONCLUSIONS  AND  RECOMMENDATIONS 

In  this  last  section  we  make  a  series  of 
suggestions  that  seem  to  follow  from  the  litera¬ 
ture.  Some  of  the  suggestions  involve  the  less- 
timid  pursuit  of  trends  and  practices  which  are 
psychologically  sound  but  only  tentatively  en¬ 
dorsed  in  the  training  community.  Other  sug¬ 
gestions  imply  research  directions;  and  still 
others  might  be  interpreted  as  criticisms  of 
directions  followed  in  the  past. 

On  Instructors 

Instructors  should  be  recruited  primarily 
(and  hopefully)  from  the  ranks  of  volunteers. 

They  should  be  demonstrably  good  at  their  aircrew 
jobs  and  be  likeable  people.  Their  final  selec¬ 
tion  should  be  recommended  by  the  pool  of  in¬ 
structors  whose  units  they  shall  join.  Their 
attitudes  about  simulation  should  be  honestly 
positive,  but  not  unrealistic.  Instructors 
should  be  trained  systematically  and  rigorously, 
covering  everything  from  full  device  utilization 
to  student  counseling.  Their  instructional 
duties  should  have  top  priority,  with  secondary 
duties  which  "come  with  the  territory"  never 
allowed  to  conflict  with  the  training  mission. 
Clearly,  there  are  serious  manpower  implications 
here,  both  in  terms  of  Tables  of  Organization 
and  the  current  military  manpower  state  of 
affairs  (i.e.,  difficulties  in  retaining  senior 
aircrewman  and  skilled  technicians). 

The  same  person  should  serve  as  instructor 
in  the  aircraft  and  in  the  simulator  for  a  given 
student.  This  reduces  nonuniformity  when  it 
comes  to  evaluating  student  progress  and  in¬ 
creases  training  standardization  (at  least  for 
his  students). 

By  no  means  should  instructional  tours  of 
duty  be  allowed  to  penalize  instructors  in  terms 
of  their  falling  behind  on  their  career  tracks 
because  of  such  factors  as  loss  of  flying  hours. 
Moreover,  special  care  needs  to  be  exercised  in 
rating  flight  instructors  by  means  of  instru¬ 
ments  weighted,  as  they  often  are,  to  a  standard 
of  mediocrity.  Never  should  the  old  saying, 
"those  who  can't  do,  teach"  be  permitted  to  become 
true  in  the  training  enterprise. 

On  Simulators 

Training  device  design  teams  should  Include 
senior  Instructors  from  the  user  population  as 
well  as  training  psychologists.  Designers  have 
access  to  a  tool  similar  to  the  Critlcality- 
Frequency-Difficulty  analysis  to  help  them  aim 
for  the  simplest  possible  training  device  needed 
to  do  the  job.  Simulators  should  be  evaluated 
primarily  for  their  teaching  qualities  and  only 
secondarily  for  their  "handling"  qualities.  It 
is  cotimon  knowledge  that  one  shortcoming  of 


simulators  is  in  reference  to  their  lack  of 
ultimate  realism.  Instead  of  berating  the 
simulator  for  its  benign,  safe,  and  relatively 
artificial  quality,  we  should  turn  the  logic 
around  and  accept  simulator  "gentleness"  as  the 
defining  positive  characteristic,  thereby 
exploiting  the  unique  qualities  of  the  simulator 
as  teaching  machine. 

We  might  even  relax  our  adherence  to  the 
training  transfer  model  which,  if  Adams  is  right, 
we  psychologists  might  have  oversold  (especially 
the  "identical  elements"  part).  Instead,  or  in 
addition,  we  ought  to  grade  our  devices  for  their 
ability  to  provide  reinforcement,  stimulus 
response  learning,  perceptual  learning,  and 
cognitive  learning. 

Apparently  too  many  simulators  are  "dropped 
over  the  fence  at  night,"  used  mainly  as  pro¬ 
cedures  trainers,  produce  too  much  hard-copy, 
break  down  too  often,  and  have  built-in  features 
of  which  too  many  users  are  kept  ignorant.  Here, 
builders  ought  to  assign  knowledgeable  field 
representatives  who  visit  simulator  sites  regu¬ 
larly  to  assure  full  and  correct  exploitation  of 
the  simulator's  features  and  to  gather  intelli¬ 
gence  for  the  next  version  of  the  device. 

We  should  proceed  with  thoughtful  efforts 
to  automate  certain  aspects  of  simulator  opera¬ 
tion,  particularly  in  the  area  of  student  perfor¬ 
mance  analysis.  It  ought  to  be  possible  for  a 
student  to  visit  the  simulator  on  his  own  time, 
insert  his  cassette  (or  performance  token  earlier 
produceTiy  that  simulator),  and  have  the  device 
tailor  a  remedial  lesson  to  that  individual  based 
upon  the  data  he  inputs. 

On  the  continuing  issue  of  flight  simulator 
motion,  perhaps  we  should  elect  a  training  mode 
(undergraduate,  transition,  or  continuation)  and 
isolate  a  few  training  units  where  long-term 
experiments  on  the  contributions  of  motion  might 
be  undertaken,  with  no-motion  assigned  to  alter¬ 
nate  students  or  classes.  An  optional  experi¬ 
mental  design,  borrowed  from  medicine,  is  a  sort 
of  rolling-block  design,  where  motion  is  inserted 
at  progressively  later  stages  in  the  training  of 
subsequent  groups  of  students.  Similar  experi¬ 
ments  could  be  done  in  the  fidelity  and  visual 
systems  areas,  with  the  aim  of  determining  the 
minimum  effective  level  of  fidelity  for  adequate 
training.  Our  design  philosophy  ought  to 
emphasize  efficiency  and  simplicity,  not  to 
mention  reliability. 

On  Instructional  Systems  Development 

A  simulator  is  a  teaching  machine,  and  a 
strong  positive  aspect  of  teaching  machines  and 
programmed  instruction  is  that  students  can  pro¬ 
ceed  at  their  own  paces.  Here,  the  plea  Is  for 
flexibility  in  syllabus  design  and  presentation 
to  enable  a  student  to  slow  down  where  needed, 
accelerate  where  appropriate,  and  skip  unneces¬ 
sary  steps.  This  flexibility  should  also  extend 
into  the  area  of  simulator  utilization  schedules 
where  down-time  is  compensated  for  not  by  jamming 
the  rate  of  fault  insertions,  but  by  solutions 
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generated  by  a  thoughtful  training  management 
team.  Too  often  simulator  utilization  schedules 
take  on  the  characteristics  of  a  five-day  tour  of 
Europe. 

Finally,  we  should  encourage  opportunities 
for  peer  training  as  well  as  self-training  as 
augmenters  of  regularly-scheduled  sessions.  Along 
these  lines  there  is  a  strong  need  for  creative 
work  in  the  area  of  off-line  post-simulator 
training  exercises.  Lately,  we  have  been  thinking 
about  something  resembling  the  athletic  practice 
of  team  viewing  and  critique  of  game  films. 

REFERENCES 

1.  Williges,  B.  H.,  Roscoe,  S.  N.,  and  Willi ges , 

R.  C.  Synthetic  flight  training  revisited. 

Human  Factors,  1973,  15(6),  543-560. 

2.  Miller,  R.  L.  Techniques  for  the  initial 
evaluation  of  flight  simulator  effectiveness. 
Wright-Patterson  AFB,  Ohio:  Air  Force 
Institute  of  Technology,  December  1976. 

3.  Fink,  C.  D.,  and  Schriver,  E.  L.,  Simulators 
for  maintenance  training:  Some  issues, 
problems,  and  areas  for  future  research. 

Lowry,  AFB,  Colorado:  Technical  Training 
Division,  Air  Force  Human  Relations  Laboratory, 
Technical  Report  AFHRL-TR- 78-27 ,  July,  1978. 

4.  Adams,  J.  A.  On  the  evaluation  of  training 
devices.  Human  Factors,  1979,  21(6),  711-720. 

5.  Caro,  P.  W.  The  role  of  simulators  in  pilot 
training:  Some  questions  regarding  effective¬ 
ness.  In  D.  S.  8ushnell,  P.  W.  Caro,  S. 
Lavisky,  H.  H.  McFann,  and  J.  E.  Taylor, 
Instructional  Technology  in  the  Military. 
Washington,  0.  C. :  First  International 
Learning  Technology  Congress,  National  Security 
Industrial  Association,  July,  1976. 

6.  Jacobs,  R.  S.,  and  Roscoe,  S.  N.  Simulator 
cockpit  motion  and  the  transfer  of  initial 
flight  training.  Proceedings  of  the  Human 
Factors  Society  19th  Annual  MeetingT  Santa 
Monica,  California:  October,  19757 

7.  Hopkins,  C.  0.  Simulators  for  training  and 
profit.  Aviation  Research  Laboratory  Tech¬ 
nical  Report  ARL-76-10/AF0SR-76-5 ,  Savoy, 
Illinois:  July,  1976. 

8.  Hopkins,  C.  0.  How  much  should  you  pay  for 
that  box?  Human  Factors,  1976,  15(6), 

533-541. 

9.  Premack,  0.  Reinforcement  theory.  In  D. 
Levine,  (Ed,),  Nebraska  Symposium  on  Motivat- 
ion  (Vol.  13).  Lincoln:  University  oY 
Nebraska  Press,  1965. 

10.  Alsobrook,  J.  E.  Attitudes  on  the  use  of 
simulators.  Maxwell  AFB,  Alabama:  Air 
Cornand  and  Staff  College,  May,  1975. 

11. 


12.  Blaiwes,  A.  S.,  Puig,  J.  A.,  and  Regan,  J.J. 
Transfer  of  training  and  the  measurement  of 
training  effectiveness.  Human  Factors ,  1973, 
15(6),  523-533. 

13.  Caro,  P.  W.,  Shelnutt,  J.  B.,  and  Spears,  W. 

D.  Utilization  of  aircrew  training  devices. 
Brooks  AFB,  Texas:  Advanced  Systems  Division, 
Air  Force  Human  Resources  Laboratory,  Tech¬ 
nical  Report  ARHRL-TR-80-  ,  February,  1980. 

14.  Charles,  J.  P.,  Instructor  pilot's  role  in 
simulation  training  (Phase  III).  Orlando, 
Florida:  Naval  Training  Equipment  Center, 
Technical  Report  NAVTRAEQUIPCEN  76-C-0034-2, 
June,  1978. 

15.  Montemerlo,  M.  0.,  Training  device  design: 

The  simulation/stimulation  controversy. 
Orlando,  Florida:  Naval  Training  Equipment 
Center,  Technical  Report  NAVTRAEQUIPCEN 
1H-287,  July,  1977. 

16.  Cream,  B.  W.,  Eggemeier,  T.  T.,  and  Klein, 

G.A. ,  A  stragegy  for  the  development  of 
training  devices.  Human  Factors,  1978,  20(2), 
145-158. 

17.  Valverde,  H.  H.  A  review  of  flight  simulat¬ 
ion  transfer  of  training  studies.  Human 
Factors ,  1973,  15(6),  510-523. 

18.  Bolwerk,  J.  M.  Navy  operational  experience 
with  flight  simulators.  Mesa,  Arizona: 

Minutes  of  Working  Group  on  Training  Simul¬ 
ation,  American  Institute  of  Aeronautics  and 
Astronautics,  October,  1979. 


ABOUT  THE  AUTHORS 

R.K.  Knapp  is  an  Associate  Professor  of  experim¬ 
ental  psychology  at  Wichita  State  University, 
while  serving  as  a  consultant  to  the  Human  Factors 
group  at  Boeing  Military  Airplane  Company.  He  has 
been  involved  in  studies  on  whole-body  vibrations, 
improved  air  refueling  systems,  trainer  technology, 
and  integrated  computer  aided  manufacturing. 

Dr.  Knapp  received  his  Ph.D.  in  experimental 
psychology  at  Michigan  State  University  in  1960. 


J.F.  Schlyer’s  training  and  work  experience  has 
been  in  the  areas  of  educational  psychology  and 
industrial  training.  He  is  presently  involved  in 
an  experimental  study  with  Dr.  Knapp  concerning 
instructor/operator  statio.  Jesigns  for  Boeing 
Military  Airplane  Company. 

Mr.  Schlyer  received  an  MS  degree  in  educational 
psychology  at  Fort  Hays  Kansas  State  College  in 
1969. 


Caro,  P.  W.  Aircraft  simulators  and  pilot 
tralnlnq.  Human  Factors,  1973,  15(6), 
502-509. 


179 


CLOSED-LOOP  TRAINING  SYSTEMS  THROUGH  THE  APPLICATION 
OF  INSTRUCTIONAL  FEATURES 


JAMES  D.  BELL 

HONEYWELL  INC.,  TRAINING  &  CONTROL  SYSTEMS  CENTER 


ABSTRACT 


Training  effectiveness  requires  that  the  training  "devices"  be  closed-loop  systems 
with  properly  applied  and  integrated  instructional  features.  This  paper  presents  a  generic 
approach  to  the  application  of  instructional  features  based  on  the  perspective  gained 
through  two  diverse  development  efforts.  The  paper  emphasizes  the  systematic  application 
of  the  techniques  such  as  cues,  prompts,  feedback  and  measurement  to  enable  an  objective 
and  appropriately  controlled  training  environment.  It  presents  for  comparison  the  results 
of  two  separate  efforts  to  develop  programmed  instruction  and  automated  performance  measure¬ 
ment  features  on  a)  the  14A2  ASW  Tactical  Team  Trainer  and  b)  the  ASN-118  Navigation 
Computer  System  (NCS)  Maintenance  Procedure  Simulator  (MPS).  A  comparative  analysis 
describes  the  difference  in  an  Instructional  features  approach  as  governed  by  the  training 
situations  that  are  supported. 

INTRODUCTION 


Rapidly  increasing  costs  of  training 
devices,  as  resources  become  more  limited,  have 
resulted  in  a  need  for  demonstrated  training  effec¬ 
tiveness  to  ensure  that  the  devices  meet  intended 
training  support  requirements.  Training  effec¬ 
tiveness  requires  that  training  devices  be  closed- 
loop  systems  with  properly  integrated  instruction¬ 
al  features.  It  is  the  use  of  these  features  that 
ensures  the  principles  of  learning  are  applied 
to  the  training  process.  This  has  not  always  been 
the  case.  It  was  noted  by  Polman  et.  al.  (1979) 
that  "advanced  control  and  display  technology  has 
been  exploited  seemingly  to  cover  all  contingen¬ 
cies  rather  than  to  permit  conduct  of  necessary 
instruction  activities  in  an  efficient  manner." 
Isley  and  Miller  (1976)  indicated  that  in  many 
modern  simulators,  incorporated  instructional 
features  were  not  even  being  used. 

The  military's  needs  for  training  systems 
are  extremely  diverse  in  nature,  including  air¬ 
craft  simulators,  portable  weapon  systems,  radar 
and  ECM  operator/team  trainers,  sonar  and  ASW  team 
trainers,  etc.  In  the  past  few  years,  training 
technology  has  been  expanded  to  include  mainten¬ 
ance  trainers  for  numerous  avionics,  weapons 
systems,  and  other  systems.  In  many  of  these 
systems,  attempts  have  been  made  to  apply  the 
technology  of  instruction  with  varying  degrees  of 
success.  The  complexity  of  applications  precludes 
a  comprehensive  discussion  of  these  approaches 
within  the  scope  of  this  paper.  However,  the 
point  is  that  an  increasing  awareness  has  grown 
for  the  need  to  advance  the  development  of  guide¬ 
lines  and  a  more  advanced  understanding  of  the 
applications  of  learning  principles  and  instruc¬ 
tional  techniques  to  the  many  diverse  training 
devices. 

Instructional  features  (automated  or 
instructor  activated  problem  control  capabilities) 
will  be  more  appropriately  integrated  into  a 
device  to  the  extent  that  they  are  viewed  as  part 
of  a  closed-loop  systei  .  The  term  "closed-loop" 
in  the  context  of  this  paper  refers  to  the  genera¬ 
tion  of  sufficient  data  to  identify  on-going 


training  problems  and  the  use  of  this  data  in 
appropriately  controlling  the  training  environment. 
Therefore,  emphasis  is  placed  on  the  application  of 
cues,  prompts,  performance  measurement  and  feedback. 
The  14A2  Team  Trainer  and  MPS  maintenance  trainer 
development  efforts  provide  excellent  examples  of 
fundamental  differences  in  training  situations 
that  are  encountered  in  the  use  of  trainers.  Some 
lessons  have  been  learned  from  these  efforts.  Based 
on  the  perspective  gained  from  these  two  efforts, 
criteria  and  rationale  for  the  application  of  cues, 
prompts,  feedback  and  performance  measurement 
instructional  features  are  described  that  are  nec¬ 
essary  to  ensure  training  effectiveness  through 
closed-loop  training  systems. 

BACKGROUND 

The  14A2  Device  Development  Effort 

The  14A2  ASW  Team  Trainer  supports  training 
of  USN  and  USCG  personnel  in  the  fundamental  pro¬ 
cedures  and  advanced  ASW  team  operations.  The  14A2 
trainer  complex,  shown  in  Figure  1,  provides  train¬ 
ing  for  the  sonar,  underwater  fire  control,  Combat 
Information  Center,  and  bridge  subteams.  Although 
both  basic  operator  and  team  training  is  supported, 
the  performance  measurement  system  development 
focused  on  team  training.  Team  training  emphasizes 
team  coordination,  and  assumes  knowledge  of  basic 
operator  procedures.  (This  does  not  imply  assumed 
proficiency).  During  training  exercises,  the  perf¬ 
ormance  of  individuals  and  subteams  has  been 
evaluated  by  use  of  subjective  checklists.  Primary 
emphasis  in  evaluation  has  been  on  overall  team 
performance.  Quantitative  performance  measures 
such  as  "number  of  targets  submarines  hit"  reflect 
this  approach.  While  such  generalized  performance 
measures  may  be  important,  they  do  not  ensure 
training  effectiveness.  When  the  target  is  not 
hit,  general  team  measures  insufficiently  define 
specific  team  weaknesses  and  strengths.  Reliance 
on  subjective  evaluation  of  key  or  critical  actions 
or  events  does  little  to  ameliorate  this  situation. 
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Figure  1.  14A2  ASW  Tactical  Team  Trainer 


The  need  for  advancements  in  team  perfor¬ 
mance  measurement  systems  has  long  been  recognized 
(Hall  and  Rizzo,  1975).  A  closely  related  problem 
is  that  of  defining  how  team  training  differs  from 
individual  training  and  the  resultant  differences 
in  performance  measurement.  In  a  study  conducted 
by  the  Navy  Training  Analysis  and  Evaluation  Group 
(TAEG)  it  was  concluded: 


The  instructor  may  select  1  of  100  distinct  simula¬ 
ted  malfunctions  in  the  NCS  as  lesson  units  in  the 
MPS.  The  trainee  conducts  checkout,  trouble  anal¬ 
ysis,  component  isolation  and  replacement,  and 
post-replacement  checkout  procedures,  in  order  to 
complete  a  complex  lesson  unit.  Extensive  use  of 
cues  and  feedback  messages  guides  the  student 
through  the  lesson. 


"There  is  much  research  to  suggest  that 
individual  proficiency  is  the  key  to 
effective  team  performance  and  that 
coordination  requi red  wi thin  a  team 
naturally  emerges  as  a  result  of  higher 
levels  of  individual  proficiency." 

In  an  approach  to  establishing  quantitative 
and  objective  performance  measures  for  the  14A2 
trainer,  Honeywell  developed  a  performance  mea¬ 
surement  paradigm  which  was  basically  in  agreement 
with  the  TAEG  group  conclusions.  The  paradigm 
referred  to  is  the  "Three  Dimensional  Matrix" 
performance  measurement  model  (Yeager  and  Bell, 
1977)  illustrated  in  Figure  2.  The  three  dimen¬ 
sions  of  the  model  are: 

1.  Subteams 

2.  Exercise  phase 

3.  Performance  measurement  categories 


A  Basis  for  Comparison  of  ASW  Team  and  Maintenance 
Training 

In  order  to  provide  a  common  basis  for 
comparison  of  ASW  team  and  maintenance  training, 
it  is  suggested  that  the  Three  Dimensional  Matrix 
performance  measurement  model  used  in  ASW  team 
training  can  be  applied  to  maintenance  training. 

In  maintenance  training,  the  model  would  be  a  sim¬ 
plified  two-dimensional  model;  categories  of  mea¬ 
surement,  and  phases.  Obviously,  the  subteams 
dimension  does  not  apply.  The  training  phases  for 
maintenance  are  checkout,  trouble  analysis,  com¬ 
ponent  isolation,  and  replacement  and  post-replace¬ 
ment  checkouts  in  that  the  student  sequences 
through  these  procedures  in  order.  The  application 
of  instructional  features  will  be  related  to  cate¬ 
gories  of  performance  measurement,  and  the  training 
situation  as  the  student  progresses  through  phases 
of  the  training  exercise. 


Individual  performance  measures  can  be 
used  to  provide  feedback  and  knowledge  of  results 
to  individual  team  members.  Scores  can  be  summed 
by  subteam,  exercise  phase  or  measurement  category. 
Composite  scores,  as  shown  in  Figure  3,  are  help¬ 
ful  in  diagnosing  specific  strengths  or  weaknesses 
by  subteam,  exercise  phase,  or  measurement  cate¬ 
gory. 

The  Three  Dimensional  Matrix  model  provided 
the  basis  for  two  related  studies  conducted  for 
the  Naval  Personnel  Research  and  Development 
Center  (NPRDC) ,  to  define  a  set  of  quantitative 
ASW  performance  measures.  In  these  stddies  a  set 
of  48  candidate  performance  measures  were  evaluat¬ 
ed  based  on  data  collected  from  six  ASW  teams 
trained  on  four  exercises  each.  Identical  cate¬ 
gory  exercises  were  selected  for  each  team.  The 
results  of  these  studies  indicate  that  approxi¬ 
mately  28  of  the  candidate  measures  proved  to  be 
meaningful  and  to  have  face  validity.  A  compre¬ 
hensive  discussion  of  the  individual  performance 
measures  and  their  application  is  provided  in  the 
resultant  NPRDC  report  (Bell,  1979).  Individual 
examples  of  measures  will  be  described  in  the 
following  sections  related  to  a  discussion  of  more 
generic  applications. 

The  MPS  Development  Effort 

The  purpose  of  the  MPS  is  to  supplement 
classroom  training  of  E-3A  aircraft  flightline 
organizational  level  maintenance  activity  for  the 
ASN-118  NCS.  The  MPS,  shown  in  Figure  4,  houses 
both  the  student  station  and  instructor  station 
in  a  single  integral  console.  The  student  station 
provides  a)  a  replication  of  the  E-3A  Navigators 
Panel,  b)  a  representation  of  the  E-3A  flight 
deck,  c)  a  trainee  panel  to  simulate  maintenance 
actions,  and  d)  a  projector  system  to  provide 
student  cues  and  error  feedback.  The  MPS  enables 
the  student  to  conduct  part  task  and  complete 
maintenance  procedure  lesson  units,  controlled  and 
monitored  with  step-by-step  programmed  Instruction 


APPLICATION  OF  INSTRUCTIONAL  FEATURES 
A  Comparison  of  the  Training  Situation 

The  first  step  in  developing  a  systematic 
approach  to  the  application  of  instructional  fea¬ 
tures  is  to  define  the  primary  characteristics  of 
the  training  situation  being  supported  by  the 
trainer.  Individual  training  features  can  then  be 
selected  on  the  basis  of  such  criteria  as  1)  com¬ 
patibility  with  the  training  situation,  2)  practi¬ 
cality,  3)  relevancy  to  the  type  of  learning  being 
addressed,  and  4)  ability  to  integrate  features 
within  the  situation.  A  summary  comparison  of  the 
instructional  situations  for  ASW  team  training  and 
E-3A  NCS  maintenance  training  is  presented  in 
Table  1.  The  differences  in  the  instructional 
situations  are  stated  in  terms  of  types  of  learning 
tasks  and  emphasis  in  training  philosophy. 

It  is  considered  beyond  the  scope  of  this 
paper  to  describe  each  difference  in  instructional 
features  in  detail.  However,  three  salient  aspects 
of  these  differences  should  be  noted: 

1.  Emphasis  with  respect  to  the  type  of 
learning. 

2.  Emphasis  with  respect  to  categories 
of  performance  (i.e.,  performance 
measurement) . 

3.  The  role  of  key  events  in  the  training 
situation  governing  the  application  of 
a  given  category  of  measures. 

Emphasis  in  training  situation  differences  and  key 
events  provided  the  basis  for  resultant  differences 
in  the  approach  to  instructional  features  in  the 
14A2  ASW  Team  Trainer  and  the  MPS  maintenance 
trainer  as  described  in  the  following  section. 
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alication  of  Cues,  Prompts  and  Feedback 


The  application  of  cues,  prompts  and  feed¬ 
back  is  primarily  influenced  by  the  type  of  learn¬ 
ing  experience  being  supported.  When  fixed  beha¬ 
vioral  {procedure  following)  sequences  are  being 
trained  the  intent  of  instruction  is  to  control 
and  shape  behavior.  Shaping  behavior  through  the 
application  of  cues,  prompts  and  feedback  in 
accordance  with  learning  principles  is  well  under¬ 
stood,  and  readily  applied.  The  maintenance  train¬ 
ing  situation  supported  by  MPS  is  well  suited  to 
the  use  of  these  features,  and  makes  extensive  use 
of  cues  and  feedback.  Appropriate  use  of  formal 
and  thematic  prompts  guide  behavior  to  ensure  that 
only  correct  produres  are  followed.  When  incor¬ 
rect  actions  are  taken,  immediate  or  delayed  feed¬ 
back  correct  the  behavior  and  reorient  the  student 
to  the  proper  stimulus  discrimination  i.e., 
interpretation  of  displays.  Cues  and  feedback  can 
be  provided  virtually  at  every  step  taken  by  the 
student  as  an  instructional  frame  or  unit. 


The  instructor  may  select  an  "aiding  level" 
that  determines  the  level  of  cues  and  feedback  to 
match  the  experience  level  of  the  student.  At 
the  "beginners”  aiding  level,  a  student  error 
results  in  an  immediate  alert  signal  and  feedback 
to  the  student;  as  well  as  subsequent  directions. 

The  "intermediate"  aiding  level  delays  feedback 
and  affords  the  student  the  chance  to  correct  the 
error  (or  repeat  the  task)  without  extrinsic  error 
feedback.  This  enables  student  behavior  to  be 
shaped  increasingly  toward  reliance  on  intrinsic 
feedback  system  status  indicators,  through  a  form 
of  diminished  intensity  cueing  and  feedback. 

Finally,  the  "test"  mode  provides  no  cues  or  feed¬ 
back,  and  only  monitors  and  records  errors. 

Several  factors  should  be  noted  about  the 
application  of  cues,  prompts  and  feedback.  First, 
it  is  assumed  that  the  training  situation  enables 
self-paced  Instruction.  Time  must  only  reflect 
proficiency  of  learning  and  should  not  be  a  cri¬ 
tical  aspect  of  the  environment  to  which  the 
trainee  must  adapt.  Second,  prompting  must  not 
detract  from  on-going,  conti nous  task  performance. 
Finally,  cues  and  feedback  must  not  interfere  with 
the  interactive  process.  Taken  in  this  context, 
cue  and  prompts  do  not  appear  to  be  compatible 
with  the  ASW  team  training  situations. 

Emphasis  on  tactical  d. ;1sion  making  in 
a  dynamically  evolving  scenario  provides  a 
radically  different  learning  experience  than  that 
in  maintenance  training.  The  trainee  must  learn 
to  act  in  a  real-time  environment.  Further, 
learning  the  tactical  consequences  of  a  bad  deci¬ 
sion,  and  how  to  recover  may  be  as  important  a 
learning  experience  as  how  to  make  the  initial 
correct  decisions.  Second,  the  trainee  must 
focus  on  developing  motor  skills  In  coordination 
with  continuous  visual  feedback  from  complex  dis¬ 
plays,  while  communicating  with  other  team  members. 
Finally,  even  in  those  cases  where  prompting  may 
benefit  one  team  member,  it  may  disrupt  the  inter¬ 
active  process,  degrading  the  training  situation 
for  other  team  members.  Therefore,  the  ASW  team 
exercise  must  be  experienced  in  Its  entirety  In 
order  that  individuals  learn  to  deal  with  evolving 
situations  in  real-time.  Individual  prompting  in 
general  is  a  disruptive  application  of  instruc¬ 
tional  features.  Feedback  should  be  provided  at 
the  completion  of  the  unit,  (I.e.,  In  post-exercise 
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evaluation).  This  will  be  discussed  in  the  follow¬ 
ing  section  in  terms  of  performance  measurement. 

It  is  still  desirable  to  avoid  “negative 
learning"  which  allows  students  to  practice 
incorrect  behaviors.  Therefore,  lack  of  guidance, 
through  prompting,  should  be  compensated  for  by 
real-time  performance  measurement  and  Instructor 
monitoring  capabilities.  Further,  the  instructor 
should  be  provided  a  manual  "freeze"  capability 
to  temporarily  halt  the  exercise  when  performance 
becomes  degraded  to  the  point  of  being  question¬ 
able.  Continous  monitoring  by  instructors  always 
carries  with  it  the  potential  for  reduced  objec¬ 
tivity  and  standardization.  Therefore,  proper 
application  of  real-time  performance  measurement 
and  problem  control  of  critical  parameters  to 
avoid  the  negative  learning  situation  are  essen¬ 
tial  to  assist  the  instructor. 


plication  of  Performance  Measures 


The  application  of  performance  measures  is 
governed  by  the  training  situation  emphasis  in 
categories  of  performance  and  key  events  in  the 
training  lesson/exercise  scenario.  Table  1  clearly 
shows  a  dichotomy  in  the  ASW  team  training  and 
maintenance  training  emphases  with  respect  to 
categories  of  performance.  In  maintenance  train¬ 
ing,  the  emphasis  is  on  procedure  following.  ASW 
team  training  places  primary  emphasis  on  communi¬ 
cations,  tactics,  and  accuracy  in  that  order. 
Performance  with  respect  to  time  is  important  in 
both  situations,  however,  the  application  of  time 
measures  is  dependent  on  key  events.  The  perfor¬ 
mance  measures  establisned  for  each  system  reflect 
the  training  situation  differences. 


The  MPS  provides  a  post-exercise  student 
performance  printout.  An  example  of  this  print¬ 
out  is  shown  in  Figure  5.  This  printout  lists 
each  error  committed  and  the  amount  of  time  into 
the  lesson  at  which  it  occurred.  The  step  number 
within  the  maintenance  Technical  Order  (T.O.) 
procedure,  criticality,  and  repeated  steps  are  all 
noted  along  with  the  total  exercise  time.  The 
printouts  are  designed  to  allow  the  instructor  to 
review  each  procedural  error,  referencing  it  to 
the  maintenance  T.O.  and  to  bring  to  the  students 
attention  specific  information,  steps,  etc.,  i.e., 
discriminative  stimuli,  to  which  the  student  must 
attend.  Time  is  strictly  an  overall  measure  of 
proficiency. 


The  development  jf  the  ASW  automated  perfor¬ 
mance  measurement  approach  addressed  several 
categories  of  pertormance  measurement.  Table  Z 
provides  a  few  examples  of  the  many  performance 
measures  determined  to  be  valid  and  meaningful  in 
evaluating  performance.  In  the  ASW  team  training 
situation,  the  least  emphasized  category  of 
measurement  is  (operational)  procedures,  the  direct 
inverse  of  the  maintenance  situation. 


In  aooltion  to  the  emphasis  placed  on  cate¬ 
gories  of  performance  measures,  the  Impact  of  key 
events  on  the  application  of  performance  measures 
is  significant.  A  comparison  of  time  measures 
provides  an  excellent  example  of  this  aspect  of 
evaluation.  In  maintenance  training  the  key 
events  are  efficient  transition  to  each  phase  of 
the  procedure.  In  general,  time  is  not  a  critical 
factor  in  performing  any  given  step.  That  is, 
the  system  state  changes  only  with  each  student 
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Date: 


Exercise  No.  149 

Aiding  Level  1 

Feedback  Node  2 

Time  Allotted  45  Minutes 

Time  Delays  Truncated 


L-.Jtep 

Error  Type 

Time 

lb 

Critical  Error 

1:15 

lb 

Repeat  Task 

1:31 

I7d 

Error 

11:10 

17d 

Repeat  Exercise 

12:24 

14c 

Error 

5:27 

24f 

Error 

17:24 

37 

Repeat  Task 

24:19 

51 

ON  FLY  FAULT  ACTIVATED 

39:12 

51 

Run  Time 

40:47 

Figure  5.  MPS  Student  Performance  Printout 


TABLE  2 

EXAMPLE  ASM  TEAM  TRAINING  PERFORMANCE  MEASURES 


Comnunications  (all  subteams) 

Number  of  Omitted  Data  Items 
Number  of  Late  Data  Items 

Accuracy 

Sonar  Range  and  Bearing  Error  (Mean,  Standard 
Deviation,  RMS) 

Doppler  Error  (Mean,  Standard  Deviation,  RMS) 
Time 

Time  to  resolve  multiple  echoes  (contacts) 
Time  to  activate  torpedo  alert 

Procedures  (all  subteams) 

Number  of  (control  positioning  errors) 


action,  and  is  not  evolving  independently.  There¬ 
fore,  time  is  consistently  a  measure  of  one  aspect 
of  performance '.proficiency.  The  total  lesson  time 
is  an  overall  measure  of  proficiency.  In  MPS 
training,  the  time  taken  to  correct  any  given 
error  is  a  measure  of  the  students  ability  to 
recognize  and  recover  from  the  error,  and  there¬ 
fore,  a  measure  of  proficiency. 

The  ASM  team  training  problem  presents  a 
more  varied  application  of  time  measures  than 
maintenance.  In  some  cases,  the  amount  of  time 
is  critical.  For  example;  the  time  taken  to 
respond  to  the  detection  of  a  hostile  weapon  fired 


at  ownship  and  alert  the  ASM  team  is  such  a  mea¬ 
sure.  In  other  cases,  the  amount  of  time  is  a 
poor  measure.  The  time  from  hostile  submarine 
detection  to  attack  is  a  measure  often  proposed, 
but  a  poor  indicator  of  performance.  In  one  case 
the  time  taken  may  reflect  proficiency  in  coordi¬ 
nating  an  attack.  In  an  other  case,  the  team  may 
be  ready  for  an  attack,  and  holding,  while  the  ship 
maneuvers  for  a  better  tactical  position.  Finally, 
time  measures  can  reflect  coordination  rather  than 
proficiency.  It  is  how  close  an  action  occurs  to 
a  key  event  rather  than  how  long  an  action  takes. 

In  an  ASW  scenario,  the  tactical  situation  is 
constantly  evolving,  partly  independent  on  any 
student  actions.  An  example  of  a  time  measure  of 
coordination  is  the  time  of  an  ordered  maneuver  by 
the  ASW  officer  based  on  relative  ownship,  assist 
ship,  and  target  position  and  motion  (key  posi¬ 
tions).  The  actual  amount  of  time  taken  to  comp¬ 
lete  the  task  is  fairly  standard.  Coordination 
is  the  key  to  maintaining  control,  and  the  learning 
experience. 

One  final  point  should  be  made  about  the 
impact  of  the  training  situation  on  performance 
measures.  It  was  noted  earlier  that  each  step  in 
maintenance  training  could  be  viewed  as  a  distinct 
instructional  frame  or  unit,  whereas  ASW  team 
training  was  a  continously  evolving  situation  that 
should  not  be  disrupted.  As  a  result,  the  MPS 
measurements  are  frequency  (number  of  occurrences) 
measures  and  time  measures.  The  ASW  performance 
measurements  deal  with  time  Intervals,  of  perfor¬ 
mance,  rather  than  discrete  points.  Therefore, 
sunmary  statistical  measures  (means,  standard 
deviations)  are  appropriate  and  often  essential 
for  evaluation  of  performance  In  a  single  exercise. 


CONCLUSIONS 
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INSTRUCTIONAL  DESIGN  FOR  AIRCREW  JUDGMENT  TRAINING 


Dr.  F.  Brecke 
Veda  Incorporated 
San  Diego,  California 


ABSTRACT 

Aircrew  training  design  has  made  significant  progress  during  recent  years.  However,  significant 
gaps  exist  both  in  design  methodology  and  existing  programs  with  respect  to  systematic  training  of 
higher  level  cognitive  skills.  Training  in  decision-making  and  judgment  is  currently  haphazard  at  best. 
After  a  brief  review  of  recent  literature,  the  paper  presents  a  conceptual  model  of  judgment  performance. 
The  theoretical  model  is  an  extrapolation  from  Jensen  (1977)  and  unites  the  variables  of  cognitive  com¬ 
plexity,  time  availability,  uncertainty,  and  stress  in  one  coherent  model.  The  model  is  used  to  examine 
current  aircrew  training  and  to  develop  new  training  strategies  for  improving  judgment  performance. 


INTRODUCTION 

The  secret  to  being  a  good  pilot  is  good 
judgment.  Very  few  people  familiar  with  the  art 
of  flying  airplanes  would  disagree  with  that 
statement.  However,  the  art  of  teaching  judgment 
to  a  pilot  is  a  secret.  Very  few  people  who  deal 
in  the  magic  arts  of  training  design  would  quarrel 
with  that  statement. 

Judgment  is  a  part  of  the  "RIGHT  STUFF"  of 
Thomas  Wolfe's  newest  novel.  A  good  student  is 
supposed  to  come  with  a  good  foundation  stock  of 
that  right  stuff.  During  training,  he  might  in¬ 
crease  it  slightly  by  mere  association  with  his 
instructors  who  have  plenty  of  it,  of  course.  But 
the  rest  of  it,  the  delta  which  will  make  him  full 
member  of  that  exclusive  elite  of  good  pilots-- 
that  delta  he  will  acquire  by  experience,  by  hours 
and  hours  of  flying,  by  countless  exposures  to 
situations  which  require  that  rare  quality  of 
good  judgment. 

The  correct  teacher,  then,  of  judgment,  is 
experience  for  us  ISD  DeoDle  who  are  so  adept  in 
teaching  facts,  concepts,  rules,  and  even  such 
high  level  skills  as  problem-solving,  grudgingly 
leave  the  field  to  Master  Experience  when  it  comes 
to  instilling  that  most  desirable  of  all  skills, 
the  skill  of  making  good  and  reliable  judgments. 
That  this  is  not  a  desirable  state  of  affairs,  and 
that  the  guild  of  instructional  designers  in  co¬ 
operation  with  the  learned  colleagues  from  the 
field  of  psychology  should  do  something  about  it, 
goes  without  saying. 

This  paper  presents  a  conceptual  framework 
for  thinking  about  the  problem  of  judgment  and  a 
discussion  of  the  applications  of  this  conceptual 
framework  to  training  design  and  research.  This 
may  sound  like  an  ambitious  endeavor,  but  if  it 
provokes  nothing  but  your  passionate  criticism, 
the  cause  of  finding  the  instructional  treatment 
for  judgment  may  be  advanced. 

The  most  recent  effort  In  the  area  of  judg¬ 
ment  training  is  a  project  sponsored  by  the  Fed¬ 
eral  Aviation  Administration  (FAA).  Phase  I  of 
this  project  resulted  in  a  report  by  Jensen  and 
Benel  (1977)  from  the  Aviation  Research  Laboratory 


at  the  University  of  Illinois.  Their  report  was 
based  on  a  literature  review  which,  to  quote  the 
authors,  "led  to  many  studies  related  to  pilot 
judgment  in  a  peripheral  way,  but  only  to  one  which 
was  directly  related"  (Thorpe,  Martin,  Edwards  and 
Eddowes  ,  1976).  The  Jensen  and  Benel  report  is 
essentially  the  first  broad-based  attempt  to  ans¬ 
wer  the  three  questions:  "What  is  judgment?", 

"Can  judgment  be  trained?",  and  finally,  "Can 
judgment  be  evaluated?”. 

The  first  question  is  answered  with  a  defi¬ 
nition  which  consisted  of  two  components: 

1)  A  cognitive  component  which  deals  with 
establishment  of  alternative  actions  and 
the  selection  among  them,  and 

2)  An  affective  component  or  motivation 
which  effects  such  selections  among 
the  alternatives. 

In  a  later  paper,  Jensen  expanded  this  defi¬ 
nition  to  include  a  continuum  from  perceptual  judg¬ 
ments  to  cognitive  judgments  (Figure  1). 


PERCEPTUAL  COGNITIVE 

JUDGMENTS  JUDGMENTS 


INCREASING  COGNITIVE 
COMPLEXITY  (AND  DECISION 


TIME) 

Figure  1.  Judgment  Continuum  Based  on  Cognitive 
Complexity  and  Decision  Time 
(Jensen,  1978) 

According  to  Jensen,  primarily  perceptual 
judgment  is  associated  with  low  cognitive  complex¬ 
ity  and  little  time  available  for  decision  making. 
At  the  other  end  of  the  continuum  one  finds  more 
analytical  forms  of  judgment,  called  cognitive 
judgment,  characterized  by  high  cognitive  complexi 
ty  and  plenty  of  available  decision-making  time. 

The  question,  "Can  judgment  be  trained?"  was 
answered  positively.  Judgment  is  trainable  if 
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Instructional  Systems  Design  (ISD)  methods  are 
used  for  training  design.  The  instructional  de¬ 
signer,  however,  finds  little  operational  advice 
which  would  enable  him  to  "do  his  thing"  in  the 
area  of  judgment,  except  the  recommendation  to  use 
situational  training  techniques. 

The  question  dealing  with  the  evaluation  of 
judgment  was  also  answered  positively.  Judgment 
performance  can  be  evaluated  if  such  an  evaluation 
is  based  on  criteria  established  by  behavioral 
objectives  and  if  the  training  situation  is  care¬ 
fully  structured.  Again,  however,  Jensen  and 
Benel  offer  no  practical  method  for  structuring 
the  training  situation. 

Neither  the  reconmendations  for  training, 
nor  the  recommendations  for  evaluation  follow  from 
the  definition.  The  definition  itself  clearly  is 
comprehensive  enough  to  encompass  all  that  might 
be  called  judgment,  but  it  is  not  sufficiently 
precise  to  allow  the  logical  deduction  of  train¬ 
ing  or  evaluation  methods. 

In  addition  to  the  Jensen  study,  there  is  a 
fairly  extensive  body  of  literature  which  deals 
with  aircrew  decision  making  in  a  probabilistic 
environment.  A  good  example  of  this  literature 
is  a  study  sponsored  by  the  Naval  Training  Equip¬ 
ment  Center  analyzing  requirements  and  prescribing 
a  methodology  for  decision  training  in  operational 
systems  (Saleh,  Leal,  Lucaccini,  Gardiner,  Hopf- 
Weichel,  1978).  Decision  making,  however,  is  only 
one,  although  the  most  complex  one,  of  the  cogni¬ 
tive  activities  to  which  the  pilot  must  apply 
judgment.  The  reconmendations  from  this  body  of 
literature,  therefore,  may  not  be  applicable  over 
the  entire  spectrum  of  the  judgment  phenomenon. 

The  training  designer,  if  he  is  not  to  leave 
the  field  of  judgment  training  to  Master  Exper¬ 
ience,  needs  a  definition  of  judgment  which  is 
both  broad  enough  and  precise  enough.  It  must  be 
broad  enough  to  Include  the  entire  judgment  spec¬ 
trum  and  it  must  be  precise  enough  to  permit  the 
deduction  of  design  principles  for  training.  In 
the  following,  an  attempt  is  made  at  a  redefini¬ 
tion  of  the  concept  of  judgment  and  the  major 
variables  which  influence  judgment. 

JUDGMENT  MODEL 

Definition 

Fundamental  to  the  understanding  of  the 
notion  of  judgment  is  the  idea  of  uncertainty  in 
the  sense  of  lack  of  information.  Judgment  must 
be  exercised  when  less-than-perfect  information 
is  available,  i.e.,  under  conditions  of  uncertain¬ 
ty.  To  express  this  in  more  colloquial  terms: 
judgment  is  making  more  or  less  educated  guesses 
if  one  does  not  know  everything  one  should  know 
in  a  given  situation,  i.e.,  when  less-than-perfect 
Information  is  available.  For  example,  the  pilot 
who  cannot  determine  how  far  the  wheels  of  his 
airplane  are  off  the  ground  during  the  final 
phases  of  his  landing  approach,  must  exercise 
judgment  in  establishing  the  correct  landing  atti¬ 
tude.  The  same  Is  true  for  a  pilot  in  an  air- 
combat  situation,  when  faced  with  an  opponent 
whose  personal  capabilities  he  does  not  know, 
whose  weapons  he  cannot  Identify,  and  who  flies 
an  obviously  modified  aircraft  of  an  otherwise 
familiar  type.  This  pilot,  too,  is  operating 


under  conditions  of  uncertainty,  i.e.,  under  con¬ 
ditions  of  a  relative  lack  of  information,  and 
must  exercise  judgment  when  he  goes  through  the 
complicated  decision-making  process  of  determining 
his  next  maneuver,  i.e.,  in  deciding  and  planning 
how  to  best  wipe  his  opponent  out  of  the  skies. 

These  two  examples,  which  range  from  simple 
to  complex  cognitive  tasks,  illustrate  another 
important  point.  This  point  is,  that  judgment  is 
not  a  type  of  intellectual  skill  as  identified, 
for  example,  in  Gagnd's  hierarchy  of  intellectual 
skills  or  in  Markle's,  Bloom’s  or  Merrill's  tax¬ 
onomies  of  objectives.  Rather,  judgment  is  exer¬ 
cised,  and  must  be  exercised,  when  performing  any 
cognitive  task,  regardless  of  its  taxonomic 
classification,  when  less-than-perfect  information 
is  available,  i.e.,  whenever  a  cognitive  task  of 
any  type  must  be  performed  under  conditions  of 
uncertainty. 

Under  conditions  of  zero  uncertainty,  with 
all  necessary  information  available,  the  mode  of 
cognitive  operation  in  any  cognitive  task  is  not 
judgment,  but  a  different  mode  of  operation  which 
one  might  simply  call  determi nation,  since  the 
outcome  is  essentially  deterministic  as  with  the 
solution  of  a  mathematical  problem.  On  the  other 
hand,  under  very  high  or  extremely  high  uncertain¬ 
ty  conditions,  the  mode  of  operation  might  more 
appropriately  be  called  intuition,  a  mode  which 
might  no  longer  be  classified  as  a  rational/cogni¬ 
tive  mode  of  operation,  as  it  borders  on  irration¬ 
al  phenomena  such  as  the  famous  “sixth  sense"  or 
ESP. 

The  concept  judgment,  therefore,  can  be  de¬ 
fined  with  three  relevant  attributes: 

•  It  is  a  mode  of  cognitive  operation 
which  is  delimited  by  strictly  deter¬ 
ministic  modes  on  the  one  side,  and  by 
Intuition  on  the  other,  where  the  bor¬ 
derline  between  judgment  and  intuition  is 
less  sharp  than  that  between  the  determin¬ 
istic  mode  of  operation  and  judgment. 

t  It  occurs  under  conditions  of  uncertain¬ 
ty. 

•  It  is  applicable  to  any  type  of  cognitive 
task,  regardless  of  its  taxonomic 
classification. 

If  this  definition  clarifies  the  concept  of 
judgment,  it  does  little  to  explain  directly  why 
different  people  faced  with  the  same  situation 
will  operate  with  different  degrees  of  success 
under  conditions  of  uncertainty,  i.e.,  will  demon¬ 
strate  anything  from  bad  to  good  judgment.  Given 
the  relative  paucity  of  the  literature  on  the 
topic  of  judgment,  it  appears  safe  to  postulate 
that  the  ability  of  a  person  to  exercise  judgment 
depends  primarily  on  four  factors  or  variables: 

1.  The  difficulty  of  the  judgment  task, 
where  the  concept  judgment  task  is 
defined  as  any  cognitive  task  which 
must  be  performed  under  uncertainty 
conditions; 

2.  The  available  repertoire  of  relevant 
cognitive  strategies,  where  cognitive 
strategies  are  defined  in  the  sense  of 
Gagnd  and  Briggs  (1974)  as  capabilities 
allowing  a  person  "to  manage  the 
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processes  of  attending,  learning, 
remembering,  and  thinking." 

3.  The  level  of  stress  at  the  moment  of 
tasking  and  the  amount  of  stress  gen¬ 
erated  by  the  task; 

4 .  The  available  repertoire  of  affective 
coping  mechanisms  for  dealing  with 
stress. 

This  appears  to  be  quite  a  plausible  set  of 
factors  which  can  be  subdivided  into  two  subsets, 
the  first  dealing  with  cognitive  factors  and  the 
second  dealing  with  affective  factors.  Face  vali¬ 
dity  of  this  set  can  easily  be  demonstrated  with 
air  combat.  Whether  or  not  a  given  pilot  will 
demonstrate  good  judgment  in  an  air  combat  situa¬ 
tion,  would  certainly  depend  on  his  mental  and 
emotional  makeup  and  the  objective,  and/or  per¬ 
ceived  danger  and  difficulty  of  the  situation  in 
which  he  finds  himself.  Physical  stress  and  the 
physical  condition  of  the  pilot  certainly  influ¬ 
ence  judgment  performance  also.  They  are  excluded 
here  because  the  intent  is  to  shed  some  light  on 
the  cognitive  and  affective  aspects  of  judgment 
training  only.  Besides  that,  the  physical  condi¬ 
tioning  part  of  pilot  training  really  does  not 
present  any  problems. 

In  the  following  now,  the  factor  difficulty 
of  the  judgment  task  is  defined  with  more  preci¬ 
sion  and  its  relationship  with  stress  is  investi¬ 
gated. 

Difficulty  of  the  Judgment  Task 

One  of  the  variables  which  determines  the 
difficulty  of  a  judgment  task  is  obviously  the 
degree  of  uncertainty.  The  higher  the  degree  of 
uncertainty,  i.e.,  the  less  information  is  avail¬ 
able  in  a  given  situation,  the  more  difficult  it 
will  be  to  exercise  judgment.  Uncertainty  and 
information  are  understood  here  in  the  sense  of 
Shannon's  Information  Theory  which  defines  infor¬ 
mation  as  a  measure  for  the  reduction  of  uncertain¬ 
ty  resulting  from  the  reception  of  a  message,  and 
which  establishes  the  bit  as  a  measure  of  infor¬ 
mation.  The  uncertainty  variable  can  thus  be 
quantified  in  bits  and  the  uncertainty  of  a  situ¬ 
ation  is  given  by  the  amount  of  information  in 
bits  that  is  necessary  for  reducing  uncertainty 
to  zero.  In  a  binary  situation,  for  example,  the 
toss  of  a  coin,  the  information  "heads"  has  the 
value  one  bit  and  reduces  uncertainty  over  the 
outcome  of  the  coin  toss  to  zero. 

A  second  variable  which  determines  the  dif¬ 
ficulty  of  the  judgment  task  was  identified  by 
Jensen  as  Cognitive  Complexity.  One  possible 
quantification  of  this  variable  is  the  number  of 
discriminators  and  operators  in  an  algorithm 
which  describes  the  cognitive  task.  For  example, 
the  task  of  identifying  a  concept  with  three 
relevant  attributes,  can  be  described  with  an  al¬ 
gorithm  containing  three  discriminators  and  three 
operators.  Saleh  et  a]_.  (1978)  have  described 
several  emergency  decision  tasks  in  algorithmic 
fashion,  using  decision  trees. 

The  third  variable  is  quite  obviously  the 
Time  Constraint,  i.e.,  the  time  available  to  per- 
form  the  task.  A  task  of  a  given  complexity  and 
uncertainty  obviously  becomes  more  difficult  the 
less  time  is  available  to  perform  it. 


For  the  sake  of  conceptual  convenience,  more 
than  for  reasons  of  theoretical  necessity,  these 
three  variables  can  be  arranged  as  the  three  axes 
of  a  spatial,  orthogonal  system  of  coordinates,  or 
as  three  orthogonal  vectors  originating  from  the 
same  point.  Difficulty  of  the  judgment  task  is 
then  simply  defined  as  the  resultant  vector. 


Figure  2.  UCT  Model:  Judgment  Task  Difficulty 
as  a  Function  of  Uncertainty,  Cogni¬ 
tive  Complexity,  and  Time  Constraint. 

For  the  sake  of  rhetorical  convenience,  the 
three-axis  model  might  be  called  the  UCT  model, 
the  acronym  being  derived  from  the  first  letters 
of  the  three  variables. 

Stress 

Before  the  application  of  the  UCT  model  to 
training  is  discussed,  the  relationship  of  judg¬ 
ment  task  difficulty  to  stress  or  to  affective 
variables  must  be  analyzed.  In  order  to  character¬ 
ize  this  relationship,  an  important  distinction, 
the  distinction  between  the  flight  problem  and  the 
background  problem,  must  be  made.  Jensen  and  Benel 
make  essentially  the  same  distinction  when  they 
speak  of  rational  and  irrational  pilot  judgment. 
This  distinction  can  best  be  demonstrated  by  an 
example  comparing  two  flights.  The  purpose  of 
Flight  No.  1  is  a  cross-country  ride  for  a  private 
pilot  license.  One  hundred  miles  before  his  des¬ 
tination,  the  pilot  receives  a  weather  report 
which  tells  him  of  an  unanticipated  deterioration 
of  the  weather  between  his  present  position  and  his 
destination,  with  his  destination  Itself  still 
under  VFR  conditions.  The  pilot  now  has  to  exer¬ 
cise  judgment  in  deciding  whether  to  continue  to 
his  destination,  to  his  preselected  alternate,  or 
to  some  other  airport,  and  whether  to  fly  through 
the  weather,  around  the  weather,  or  above  the 
weather.  In  Flight  No.  2,  exactly  the  same  situa¬ 
tion  exists.  One  hundred  miles  before  his  destina¬ 
tion,  the  pilot  receives  the  very  same  weather  re¬ 
port.  He  flies  the  same  aircraft,  etc.  However, 
the  purpose  of  this  flight  Is  not  a  cross-country 
check  ride,  but  an  Important  business  meeting,  or 
a  reunion  with  a  lover,  or  a  medical  emergency. 
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In  Flight  No.  1,  the  pilot  is  faced  with  solving  a 
pure  flight  problem.  He  will  exercise  his  best 
judgment,  given  the  facts  and  uncertainties  of  the 
flight  situation  alone.  In  Flight  No.  2,  the  same 
flight  problem  exists,  but  in  addition  to  that,  a 
second  problem  which  can  be  called  the  background 
problem,  is  to  be  solved.  The  background  problem 
and  the  flight  problem  obviously  interact  with  each 
other.  It  can  be  hypothesized  that  if  this  inter¬ 
action  between  these  two  problems  produces  conflict 
and/or  cognitive  dissonance  (as  defined  by  Festin- 
ger),  a  component  of  stress  is  introduced  into  the 
situation  which  can  be  considered  additive  to  the 
stress  already  existing  at  the  time  the  problem 
emerged. 

Stress,  however,  arises  not  only  from  con¬ 
flict  created  by  the  interaction  between  the  flight 
and  the  background  problem,  but  also  from  the 
flight  problem  itself.  The  difficulty  of  the  judg¬ 
ment  task  in  the  pure  flight  problem  may  also  give 
rise  to  stress.  The  relationships  between  stress 
and  task  difficulty  have  been  extensively  investi¬ 
gated  in  human  factors  research,  and  by  and  large 
allow  the  conclusion  that  performance  will  initial¬ 
ly  rise  with  increasing  stress  up  to  a  certain 
point,  where  it  rapidly  drops  off  to  near  zero 
level.  Among  flight  instructors  in  military  flight 
training,  this  phenomenon  is  well  known  as  the  so- 
called  "IQ  Dump".  There  are  probably  very  few 
pilots,  especially  in  military  aviation,  who  have 
not  at  one  time  or  another  experienced  this  pheno¬ 
menon  themselves,  either  in  training  or  during 
operational  flying.  By  the  way,  military  aviation 
is  emphasized  here,  not  because  it  is  assumed  that 
the  men  (and  nowadays  women)  who  choose  to  become 
military  pilots  are  especially  likely  to  "dump" 
their  IQs,  but  because  military  aviation  is  es¬ 
pecially  demanding. 

Stress  can  thus  be  seen  as  a  fourth  vector 
in  the  model,  adding  an  affective  component  to  the 
definition  of  judgment  task  difficulty  (Figure  3). 


Figure  3:  UCT  Model:  Stress  as  the  Fourth 
Vector  Determining  Judgment  Task 
Difficulty  (D). 


So  far,  then,  two  of  the  four  factors  likely 
to  influence  judgment  performance  have  been  dis¬ 
cussed.  The  following  conclusions  seem  warranted: 

•  Judgment  task  difficulty  can  be  seen  as 
the  resultant  vector  of  cognitive  com¬ 
plexity,  uncertainty,  and  the  inverse 
of  time  availability. 

•  Stress  will  affect  judgment  performance 
in  a  non-linear  fashion:  positively  up 
to  an  individual  maximum,  and  negatively 
beyond  that.  The  stress  in  a  situation 
requiring  judgment  can  be  thought  of  as 
consisting  of  three  components:  the 
null -level  stress  or  the  stress  at  the 
point  of  tasking,  stress  resulting  from 
the  difficulty  of  the  judgment  task 
itself,  and  stress  resulting  from  the 
interaction  of  the  flight  problem  and 
the  background  problem. 

Types  of  Judgment  Tasks 

Before  discussing  the  application  of  the 
model,  one  final  distinction  should  be  introduced. 
It  is  possible  to  distinguish  between  two  types 
of  judgment  tasks.  The  first  type.  Type  1,  occurs 
when  the  flow  of  some  routine  evolution  is  dis¬ 
turbed  by  some  unpredicted,  unforeseen  occurrence 
(such  as  in  an  emergency  or  malfunction),  or  when 
something  planned  and  predicted  has  not  occurred 
(such  as  when  the  tanker  aircraft  does  not  show 
up  at  the  planned  rendezvous).  Many  such  situa¬ 
tions  are  covered  by  procedures,  emergency  and 
otherwise,  but  the  potential  variations  in  the 
situations  are  so  innumerable  that  it  would  not 
only  be  inefficient,  but  virtually  impossible,  to 
invent  a  procedure  for  every  eventuality. 

The  second  type  of  judgment  tasks  are  those 
tasks  where  the  course  of  events  is  never  routine, 
never  predictable,  always  uncertain,  and  where 
judgment  is  therefore  always  required.  This  type 
of  judgment  task  may  be  called  a  Type  2  task  and 
air  combat  maneuvering  is  a  perfect  example  for 
this  class  of  judgment  tasks. 

APPLICATION  OF  THE  MODEL 

The  two  remaining  factors,  availability  of 
relevant  cognitive  strategies  and  availability  of 
relevant  affective  strategies  for  dealing  with 
stress,  will  be  discussed  in  the  dual  context  of 
selection  and  training. 

Selection 

In  the  selection  process  for  training  in  the 
Israeli  Air  Force,  the  applicants  are  subjected  not 
only  to  paper-and-pencil  tests  and  personality 
interviews,  but  also  to  situational  tests  which 
tax  an  applicant's  intellectual  resourcefulness 
and  his  ability  to  withstand  psychological  stress 
in  practical,  hands-on  situations.  It  appears  that 
these  selection  procedures  are  quite  effective  and 
that  their  effectiveness  can  be  directly  understood 
in  view  of  the  preceding  discussion.  Given  the 
conceptual  model  above,  it  would  appear  possible 
to  design  carefully  graduated  situations  varying 
in  cognitive  complexity,  uncertainty,  time  availa¬ 
bility,  and  stress  that  would  yield  valid  predic¬ 
tive  scores,  when  referenced  to  scores  of  a  norm 
group  of  successful  pilots. 
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Training 

If  the  conceptual  model  presented  above  is 
applied  to  training,  one  can  draw  conclusions  con¬ 
cerning  the  validity  of  current  aircrew  training 
methods  and  one  can  begin  to  define  new  training 
strategies  specifically  aimed  at  the  development  of 
good  judgment  performance  under  various  degrees  of 
psychological  stress. 

Current  military  aircrew  training  is  charac- 
terized  by  an  emphasis  on  correct  completion  of 
prescribed  procedure,  compliance  with  rules,  and 
specific  flight  techniques.  Judgment  is  a  coveted 
quality  and  is  usually  evaluated  under  such  head¬ 
ings  as  "headwork"  or  "airmanship",  but  explicit 
instruction  in  headwork  or  airmanship  is  almost 
totally  absent. 

The  author  has  personally  received  this  type 
of  training  in  the  U.S.  Air  Force  and  can  attest 
to  its  inadequacy  based  on  his  rude  awakening  upon 
assignment  to  an  operational  squadron  of  the  Ger¬ 
man  Luftwaffe.  A  former  colleague  of  mine  and 
former  F-14  instructor  pilot,  describes  current 
training  practices  in  the  Navy  as  follows: 

"The  student  is  taught  from  day  one, 
that  he  must  use  correct  procedures. 

He  is  drilled  for  hours  in  these  pro¬ 
cedures  and  learns  to  accomplish  tasks 
using  the  same  steps  each  and  every 
time.  Use  of  checklists  to  cover  nor¬ 
mal  ,  abnormal ,  and  emergency  procedures 
is  mandatory. 

At  the  same  time,  he  is  presented 
with  a  myriad  of  rules  to  follow. 

Compliance  with  Course  Rules,  Air 
Traffic  Control,  NATOPS,  SOP,  as 
well  as  Station,  Wing,  and  NAVAIR 
instructions  is  taught. 

When  reasonably  proficient  in  appli¬ 
cable  procedures  and  rules,  the  stu¬ 
dent  is  taught  flying  technique  and 
is  evaluated  on  "basic  airwork".  If 
the  student  has  an  opportunity  to 
exercise  judgment,  he  is  evaluated 
under  the  heading  of  "headwork". 

The  system  is  structured,  however, 
so  that  virtually  no  opportunity 
exists,  under  normal  circumstances, 
for  a  student  to  exercise  his  judg¬ 
ment.  The  student  will  be  success¬ 
ful  if  he  follows  correct  procedures 
and  rules. 

Fleet  replacement  squadrons  follow 
a  similar  approach  to  aircrew  train¬ 
ing.  Flight  techniques  (including 
weapon  system  utilization)  are  more 
heavily  emphasized  than  procedures 
or  rules  as  a  student  is  expected 
to  master  procedures  and  rules 
quickly  with  a  high  degree  of  trans¬ 
fer  of  knowledge  from  the  Training 
Command. 

Graduates  of  the  present  system  can 
be  characterized  as  reasonably  compe¬ 
tent  airmen,  well -Indoctrinated  in 
applicable  procedures  and  rules, 
whose  decision-making  abilities  are 
relatively  unknown. 


Once  in  the  fleet,  the  fledgling 
aviator  is  expected  to  learn  by 
example  as  he  flies  with  an  exper¬ 
ienced  flight  leader  and/or  crewman. 

When  a  situation  arises  requiring 
exercise  of  judgment,  he  can  normally 
count  on  being  assisted  by  many  ex¬ 
perienced  people,  both  in  the  air  and 
on  the  ground. 

Problems  arise  in  this  system  when  the 
inexperienced  aviator  is  cut  off  from 
the  decision-making  assistance  of  others. 

The  student  has  very  little  to  fall  back 
on,  except  his  procedures  and  rules.  Un¬ 
fortunately,  there  are  not  procedures  to 
cover  every  problem  that  may  arise. 

The  aviator  will  probably  decide  on  an 
acceptable  course  of  action  based  on  his 
knowledge  and  the  decision-making  skills 
and  judgment  that  he  developed  prior  to 
Navy  flight  training. . . " 

When  this  state  of  affairs  is  examined  in 
light  of  the  model  and  the  concepts  presented  pre¬ 
viously,  it  is  obvious  that  training  in  Type  1 
judgment  tasks  is  totally  absent  or  accidental  in 
two  respects:  it  occurs  only  by  accident,  and 
when  it  occurs  in  the  air,  it  is  likely  to  cause 
an  accident.  Training  in  Type  2  judgment  tasks 
exists  only  in  the  form  of  feedback,  such  as  in 
the  ACMR  facilities  when  an  air  combat  maneuvering 
sortie  is  replayed  in  three  dimensions  and  each 
move  and  countermove  is  discussed  post-facto. 

Is  explicit,  systematic  training  of  judgment 
really  all  that  important?  After  all,  the  train¬ 
ing  pipelines  do  produce  aviators  who  are  able  to 
function  successfully  in  peacetime  as  well  as  in 
combat.  The  counter  question  here  is:  "At  what 
cost?”  It  is  well  known,  from  military  as  well 
as  from  civilian  accident  statistics,  that  the 
overwhelming  majority  of  aircraft  accidents  is 
attributable  to  pilot  error  and  that  most  of  these 
are  simply  errors  in  judgment.  In  other  words: 
waiting  for  experience  to  teach  you  enough  judg¬ 
ment  can  kill  you. 

Further  insight  into  the  importance  of  ex¬ 
plicit  and  systematic  judgment  training  throughout 
the  pilot's  career  is  gained  when  comparing  jobs 
which  are  highly  proceduralized  by  their  very  na¬ 
ture,  such  as  the  job  of  a  bank  clerk  with  the  job 
of  a  military  combat  pilot.  Figure  4  illustrates 
the  task  universes  of  these  two  jobs.  They  are 
arbitrarily  limited  to  intellectual  tasks  using 
Gagnd's  taxonomy. 

This  illustration,  by  the  way,  is  not  based 
on  actual  data,  but  rather  on  sound  judgment.  The 
intent  is  to  illustrate  a  principle  rather  than  to 
provide  exact  quantitative  data.  The  fact  that 
the  two  task  universes  are  different  in  overall 
size  is  irrelevant.  What  is  relevant,  however, 
is  the  fact  that  the  task  universe  of  the  pilot 
contains  much  higher  proportions  of  high-level 
tasks  and  much  higher  proportions  of  judgment 
tasks,  i.e.,  incidents  where  tasks  must  be  per¬ 
formed  under  conditions  of  uncertainty.  There  is 
little  doubt  in  a  bank  clerk's  job,  once  he  has 
learned  his  rules  and  procedures.  That  is  not  the 
case  in  a  pilot's  job.  When  viewed  in  this  light, 
current  training  practices  essentially  provide  the 
pilot  with  bank  clerk  training.  This  is  hardly  in 
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accordance  with  the  much  touted  maxim  "Train  Like 
You  Fight". 


j  j  fMtt 


Figure  4.  Proportions  of  Judgment  vs.  Non- 

Judgment  Tasks  in  2  Task  Universes. 

if  a  fighter  pilot,  or  any  other  military 
pilot  for  that  matter,  must  have  one  characteris¬ 
tic,  one  trait  that  is  absolutely  essential  in  the 
performance  of  his  duties,  that  trait  is  probably 
initiative  (besides  judgment).  However,  trainees 
who  are  continuously  given  directive  commentary 
and  who  are  continually  drilled  in  rigid  rules  and 
procedures  will  hardly  develop  initiative,  but 
much  more  likely  will  abdicate  responsibility  and 
await  orders.  Training  in  judgment  on  the  other 
hand,  i.e.,  training  under  conditions  of  uncer¬ 
tainty,  especially  if  such  training  follows  an 
orderly  step-by-step  course,  systematically  devel¬ 
ops  a  trainee's  self-reliance  and  with  that,  his 
initiative. 


One  more  point  should  be  made  regarding  the 
importance  of  including  judgment  training  in  air¬ 
crew  training  curricula.  The  current  rigid  regi¬ 
mentation  and  procedural ization  of  training  fre¬ 
quently  leads  to  actual  fear  of  departing  from 
established  procedures.  Not  all  emergency  situa¬ 
tions  fit  precisely  the  case  for  which  a  given 
procedure  is  designed.  As  a  consequence,  the 
rigidly  trained  aviator  who  is  unable  to  and  fear¬ 
ful  of  bending  the  procedure  might  end  up  bending 
the  airplane. 


Given  these  arguments,  the  next  logical 
question  is:  "If  judgment  training  is  so  impor¬ 
tant,  then  why  is  it  not  included  in  the  average 
aircrew  training  curriculum?".  Two  reasons  come 
to  mind.  First  of  all,  it  is  much  easier  to  train 
overt,  observable  behavior.  This  state  of  affairs 
has  not  changed  with  the  advent  of  ISO.  Task 
analyses  and  objectives  hierarchies  for  aircrew 
training  rarely  contain  the  covert,  higher  level 
cognitive  tasks  required  of  the  coirfcat  pilot. 


The  second  reason 
enon  where  each  accident 
procedure  to  be  included 


is  the  well  known  phenom- 
spawns  another  rule  or 
in  the  fl ight  manual . 


The  attempt  to  reduce  costly  errors  due  to  bad 
judgment  by  regulation  and  procedural ization  is 
like  trying  to  cure  the  problem  by  treating  the 
symptoms  instead  of  the  causes.  In  the  words  of 
one  instructor  pilot:  "The  exercise  of  judgment  is 
inversely  proportional  to  the  amount  of  regulation. 
The  amount  of  regulation  is  usually  proportional  to 
the  cost  of  judgment  errors."  Or,  to  say  it  some¬ 
what  differently:  The  attempt  to  remedy  judgment 
errors  by  procedural ization  starts  a  vicious  circle 
which  only  leads  to  more  errors  in  judgment.  The 
only  cure  is  to  train  judgment  systematically. 

New  Training  Strategies 

The  model  presented  in  this  paper  offers  a 
methodology  by  which  such  systematic  training  can 
be  conceptualized,  designed,  and  implemented. 

First  of  all,  judgment  training  '-ust  enable 
the  trainee  to  distinguish  the  flight  problem  from 
the  background  problem.  As  mentioned  before,  back¬ 
ground  problems  may  include  such  things  as  medical 
emergencies,  business  trips,  and  lovers.  However, 
they  can  be  considerably  more  subtle  than  that. 

One  background  problem  that  the  military  pilot  is 
especially  prone  to  encounter  is  excessive  machismo, 
brought  about  by  peer  group  pressure.  This  brings 
to  mind  a  famous  Royal  Air  Force  proverb:  "There 
are  old  pilots  and  there  are  bold  pilots,  but  there 
are  no  old  bold  pilots."  The  "can  do"  attitude 
that  is  especially  prevalent  in  today's  Navy,  quite 
likely  fosters  a  scarf-flying,  devil-may-care  atti¬ 
tude  rather  than  good  and  sound  judgment.  Given  a 
behavioral  objective  such  as  "The  student  will  iden¬ 
tify  the  background  problem,  given  verbal  descrip¬ 
tions  of  flight  scenarios,  correctly  in  x  out  of  y 
cases,"  it  should  not  be  too  difficult  to  design 
appropriate  instruction.  Such  instruction  would 
generate  in  the  student  an  awareness  of  how  the 
background  problem  may  bias  or  interfere  with  his 
judgment  in  solving  the  flight  problem. 

Secondly,  the  three-pronged  model  of  uncer¬ 
tainty,  cognitive  complexity  and  time  availability, 
allows  systematic  design  of  scenarios  which  are 
carefully  graduated  along  these  three  variables. 

If  the  student  is  subjected  to  successively  more 
difficult  judgment  problems  under  gradually  in¬ 
creasing  psychological  stress,  and  if  all  such 
training  sequences  are  administered  on  the  basis  of 
reaching  criterion  performance  before  progressing 
to  the  next  higher  level  of  task  difficulty,  it 
would  appear  reasonable  to  expect  a  high  degree  of 
training  effectiveness. 

For  example,  in  any  given  area  of  flight 
instruction,  be  it  familiarization,  navigation, 
air-to-ground  tactics,  air-to-air  tactics,  etc., 
the  model  allows  the  design  of  a  variety  of  poss¬ 
ible  systematic  training  strategies  which  lead 
from  the  least  difficult  to  the  most  difficult  of 
judgment  problems.  One  such  possible  strategy  is 
presented  in  Figure  5  in  the  form  of  a  series  of 
histograms  which  show  relative  quantities  of  the 
four  variables  of  uncertainty,  cognitive  complexity, 
time  availability  and  stress. 

As  the  figure  shows,  the  student  is  initially 
presented  with  scenarios  or  situations  of  low  cog¬ 
nitive  complexity  and  ample  time  availability.  As 
he  becomes  proficient,  an  element  of  uncertainty  is 
introduced  while  keeping  complexity  and  time  con¬ 
stant.  Gradually,  stress  is  Introduced,  the  time 
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Figure  5.  Histogram  Representation  of  an  8-Step 
Strategy  for  Judgment  Training. 

available  is  decreased,  and  the  levels  of  uncer¬ 
tainty  and  cognitive  complexity  are  built  up  until 
the  student  can  finally  demonstrate  acceptable 
judgment  performance  over  the  entire  spectrum  of 
the  particular  subject  matter  domain  being  trained. 

The  gradual  build-up  of  stress  deserves 
special  attention.  Classroom  instruction,  i.e., 
the  academic  environment,  presents  very  little 
natural  stress.  Stress  is  somewhat  increased  when 
the  student  transitions  from  the  classroom  to  the 
simulator  environment,  since  he  is  there  operating 
under  real-time  conditions  and  under  the  influence 
of  much  more  sensory  stimulation  than  in  the  class¬ 
room.  The  transition  from  the  simulator  environ¬ 
ment  to  the  flight  environment,  however,  represents 
a  quantum  jump  in  stress.  The  reason  for  this  is 
quite  simply,  fear  of  death.  The  simulator  cannot 
crash.  It  is,  therefore,  important  that  the  stu¬ 
dent  is  accommodated  to  these  higher  levels  of 
stress  in  the  preceding  simulator  and  academic  en¬ 
vironments.  It  is  equally  important  not  to  overdo 
this  principle  so  that  the  student  does  not  devel¬ 
op  fear  of  flying. 

A  second  note  of  caution  should  be  intro¬ 
duced  here.  Current  training  essentially  remains 
in  the  deterministic  plane  {Figure  6).  The  train¬ 
ing  situations  which  are  administered  are  usually 
canned,  and  have  very  little  built-in  uncertainty. 
As  mentioned  previously,  the  student  learns  in¬ 
numerable  rules  and  procedures.  On  the  face  of  it, 
it  would  seem  quite  reasonable  to  retain  this  sort 
of  training  and  to  proceed  along  the  uncertainty 
axis  only  after  the  student  has  acquired  a  firm 
basis  in  the  deterministic  plane.  One  could,  for 
example,  use  in  tactical  decision-making,  highly 
complex  algorithms  that  the  student  works  through 
in  a  classroom  environment.  It  is  doubtful  whether 
training  in  such  deterministic  modes  of  operation 
has  a  high  degree  of  transfer  to  judgment  perform¬ 
ance  which  is  an  essentially  different  mode  of 
cognitive  operation,  i.e.,  a  probabilistic  mode. 
Flying  will  never  be  totally  predictable,  and  any 
attempts  to  transform  the  pilot  into  a  high-speed 


Figure  6.  UCT  Model:  The  Deterministic  Plane 

automaton  or  computer  are  doomed  to  failure.  The 
development  of  cognitive  strategies  that  are  likely 
to  be  successful  in  an  uncertain  environment  is 
probably  fostered  more  effectively  and  efficiently, 
if  uncertainty  is  introduced  all  along,  i.e.,  if 
the  student  is  led  along  a  path  which  proceeds 
step-wise  along  each  of  the  axes. 

Research 

The  preceding  discussion  of  training  design 
strategies  based  on  the  UCT  model  immediately  leads 
to  numerous  questions  which  can  only  be  solved  by 
experimentation.  It  is  suggested  that  the  UCT 
model  is  an  exceptionally  productive  coi  ceptual 
vehicle  for  the  design  of  research  programs  because 
of  its  face  validity  on  the  one  hand,  and  because 
of  the  many  questions  it  poses,  on  the  other  hand. 

It  appears  suitable  not  only  as  a  vehicle  for  in¬ 
vestigating  instructional  strategies  for  judgment 
training,  but  also  for  the  investigation  of  the 
judgment  phenomenon  itself.  The  aircrew  training 
community,  however,  can  hardly  afford  to  wait  until 
the  results  from  such  a  research  program,  which 
would  take  years  to  put  into  motion  and  years  to 
conduct,  are  available.  Such  waiting  would  only 
perpetuate  the  depressing  accident  statistics  on 
pilot  error  and  endanger  the  combat  readiness  of 
our  Armed  Forces. 

Beginnings 

The  aircrew  training  community  has,  in  fact, 
already  begun  to  implement  new  forms  of  training 
which  are  likely  to  foster  the  development  of  good 
judgment.  For  example,  the  F-15  program  features 
situational  emergency  training.  In  this  program, 
the  old  style  boldface  emergency  procedure  train¬ 
ing  is  totally  absent.  The  student  learns  only 
three  rules:  1.  maintain  aircraft  control,  2. 
analyze  the  situation  and  take  proper  action,  and 
3.  land  as  soon  as  practicable.  The  student  is 
presented  with  carefully  structured  scenarios, 
both  in  the  classroom  and  the  cockpit  procedures 
trainer,  and  is  consistently  encouraged  to  use  his 
judgment  in  finding  the  proper  action  for  his  problem. 
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Another  example  is  the  air  combat  training 
syllabus  for  the  F- 14  aircrews,  developed  by  Veda 
Incorporated.  In  this  syllabus,  the  pilots  are 
first  trained  in  the  classroom  in  the  general  prin¬ 
ciples  of  maneuvering  an  F-14  in  an  air  combat  sit¬ 
uation.  They  then  proceed  to  the  ACMR  debriefing 
facility  which  is  used  in  a  pro-active  manner, 
rather  than  in  the  debriefing/feedback  mode.  The 
student  is  there  presented  with  a  graphic  image  of 
a  typical  engagement,  and  on  a  second  screen,  with 
such  data  as  airspeed,  altitude,  etc.,  for  both  the 
fighter  and  the  opponent.  The  dynamic  graphic  pre¬ 
sentation  is  frozen  at  crucial  points,  and  the 
students  are  asked  to  assess  the  situation  either 
from  the  standpoint  of  the  fighter  or  the  opponent, 
and  to  exercise  their  judgment  in  deciding  on  their 
next  move.  It  is  quite  possible  (however,  not  pre¬ 
sently  implemented)  to  create  fairly  high  levels 
of  stress  by  increasing  the  pressure  for  quick 
answers  and  by  punitive  comments  for  wrong  or  slow 
answers.  The  students  then  transfer  to  the  simu¬ 
lator  environment  where  they  fly  the  same  types  of 
engagements.  The  simulator  of  course  also  offers 
the  advantage  of  decision  time  manipulation  by  the 
freeze  capability.  Stress  levels  again  can  be 
manipulated  by  the  manner  in  which  the  instructor 
reacts  to  the  student’s  actions.  The  student 
finally  proceeds  to  the  flight  environment,  well 
prepared  for  the  split-second  judgments  he  must 
make  under  stress.  He  no  longer  finds  himself 
thrust  suddenly  into  an  environment  which,  compared 
to  his  preceding  instruction,  shows  simultaneously 
enormously  increased  levels  of  uncertainty,  com¬ 
plexity  and  stress,  and  much  less  available  time. 
The  phenomenon  of  the  IQ  dump  will  hopefully  occur 
with  less  frequency,  but  it  is  as  yet  too  early 
to  tell . 

SUMMARY 

In  his  recently  published  work,  "The  Third 
World  War",  General  Sir  John  Hackett  advances  the 
hypothesis  that  said  world  war  will  be  won  by-- 
among  other  things--the  higher  degree  of  indepen¬ 
dence  in  junior  leaders  in  the  tactical  area.  Such 
independence  requires  judgment,  good  judgment  under 
stress.  There  can  be  little  doubt  about  the  impor¬ 
tance  of  systematic  judgment  training  not  only  for 
aircrews,  but  throughout  the  Armed  Forces. 

This  paper  suggested  a  conceptual  model 
and  a  design  approach  for  systematic  judgment 
training.  The  ideas  presented  here,  albeit  not 
tested  by  either  experiment  or  experience,  are 
simple  and  operational  enough  to  be  put  into  prac¬ 
tice  inroediately. 
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ABSTRACT 

This  paper  describes  an  applied  training  research  study  in  the  use  of  computer  simulation 
being  conducted  for  the  Chief  of  Naval  Education  and  Training  (CNET)  by  the  Training  Analysis 
and  Evaluation  Group  (TAEG)  in  the  U.S.  Navy.  The  study  is  a  part  of  the  Consolidated  Navy 
Electronic  Warfare  School  (CNEWS)  training  system  program.  A  computer  simulation  model  has  been 
developed  to  model  the  complex  student  flows  and  training  resource  utilization  patterns  through 
a  multi-track,  mixed  group-paced  and  self-paced  training  curriculum  which  exists  in  the  Electro¬ 
nic  Warfare  School  (EW)  at  the  Naval  Technical  Training  Center.  This  paper  also  describes  the 
effectiveness  of  using  SLAM  as  a  simulation  language  to  model  the  network-oriented  structure  of 
this  simulation. 

I.  INTRODUCTION 


Background 

The  Consolidated  Navy  Electronic  Warfare 
School  (CNEWS)  at  the  Naval  Technical  Training  Cen¬ 
ter  (NAVTECHTRACEN) ,  Corry  Station,  Pensacola, 
Florida,  conductes  basic  and  advanced  operator  and 
maintenance  training  for  25  types  of  students, 
including  officers  and  enlisted,  and  includes  more 
than  14  separate  student  pipelines  with  various 
combinations  of  courses  of  training. 

In  May  1977,  Chief  of  Naval  Education  and 
Training  (CNET)  assigned  TAEG  the  responsibility 
for  the  development  of  a  new  consolidated  EW  oper¬ 
ator  training  curriculum  for  CNEWS.  Once  implemen¬ 
ted,  the  new  operator  training  system  will  provide 
individualized  Instruction  using  a  variety  of  train¬ 
ing  resources  such  as  programmed  instruction;  nar¬ 
rative  texts,  sound-slide  programs;  random-access, 
interactive  videotape  programs;  classroom  and  labo¬ 
ratory  Instruction;  training  devices  and  operational 
equipment.  The  curriculum  will  be  managed  by  means 
of  the  Navy's  Computer  Managed  Instruction  (CMI) 
system. 

In  the  new  curriculum,  all  students  will  not  take 
all  learning  modules,  but  will  proceed  through  the 
respective  learning  tracks  tailored  to  their  speci¬ 
fic  Instructional  needs  and  their  next  duty  assign¬ 
ments.  A  central  component  of  the  EW  operator 
training  system  will  be  a  60-student  station  gener¬ 
alized  EW  operatory  training  device,  Device  10H1. 

The  device  provides  computer-aided  training  in 
general  system  familiarization,  operator  skills  de¬ 


velopment,  operating  techniques,  EW  capabilities 
and  limitations  and  EW  watch-standing  and  tactical 
exercises. 

As  the  development  of  the  EW  operator  curri¬ 
culum  progressed,  it  became  apparent  that  the  com¬ 
plexity  of  the  task  dictated  the  need  for  a  manage¬ 
ment  planning  tool  for  CNEWS  planners  to  aid  them 
in  early  identification  of  the  likely  impacts  of 
the  new  curriculum  on  the  other  maintenance  and 
operator  training  courses  currently  being  taught 
and  those  that  will  continue  to  be  taught  at  CNEWS 
after  the  individualized  curriculum  is  on  line. 

The  problem  confronting  TAEG  and  the  CNEWS 
planners  is  twofold;  i.e.,  (1)  providing  group- 
paced  instruction  in  a  variety  of  operator,  main¬ 
tenance,  and  technology  courses  using  the  school's 
current  resources;  and,  (2)  concurrently  planning 
for  the  development  and  implementation  of  Device 
10H1  and  the  development  and  implementation  of  the 
new  individualized  EW  operator  curriculum. 

Compounding  the  CNEWS  planning  problem  is  the 
fact  that  student  input  rates  fluctuate  over  time. 
Therefore,  the  school's  operation  will  have  to  be 
periodically  adjusted  to  accommodate  increased  and 
decreased  student  input  rates  in  order  to  supply 
adequate  quantities  and  appropriate  types  of  train¬ 
ing  materials,  equipment,  media,  and  training  de¬ 
vices  (such  as  an  adequate  number  of  student  sta¬ 
tions  for  Device  10H1).  These  adjustments  will  be 
necessary  to  assure  smooth  student  flow  through  the 
curriculum  in  order  to  meet  student  output  require- 
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ments. 


Solving  the  forecasting,  planning,  scheduling, 
and  resource  management  problems  facing  CNEWS  plan¬ 
ners  (significantly  complicated  by  the  implementa¬ 
tion  of  the  new  EW  operator  curriculum  and  the  in¬ 
troduction  of  Device  10H1)  requires  assessment  of 
numerous  integrated  factors.  The  character  of  the 
problem  precludes  easy  solution  by  traditional 
means.  This  type  of  problem  is,  however,  ideally 
suited  to  solution  by  computer  simulation. 

The  objective  of  this  study  is  to  develop  a 
user-oriented,  predictive  computer  model  of  the 
CNEWS  student  flow  through  the  various  courses  in 
its  curriculum.  The  model  should  allow  the  EW 
planner  to  manipulate  and  evaluate  various  curri¬ 
culum  variables  via  simulation  before  committing 
such  approaches  to  development  and  implementation. 

II.  CONSOLIDATED  NAVY  ELECTRONIC  WARFARE  SCHOOL 

This  section  describes  CNEWS’  current  and  pro¬ 
jected  training  program.  The  training  program 
description  includes  a  discussion  of  student  input 
patterns,  EW  course  descriptions,  student  flows 
(tracks)  through  courses  in  the  curriculum,  train¬ 
ing  facility,  media/device/operation  equipment 
utilization  patterns,  and  school  training  manage¬ 
ment  policies. 

Three  major  types  of  students  attend  CNEWS; 
i.e.,  officers,  enlisted  EW's  and  enlisted  person 
nel  from  other  Navy  ratings  and  other  branches  of 
the  military.  Some  civilians  and  foreign  military 
personnel  undergo  training  at  CNEWS,  but  their 
number  is  small,  and  for  purposes  of  model  develop¬ 
ment,  they  are  not  included  in  this  study.  In 
order  to  describe  CNEWS'  current  and  projected 
training,  it  is  necessary  to  define  the  basic  ter¬ 
minology  and  data  base  elements  adopted  in  the 
study. 

CNEWS  Data  Base  Elements 

The  first  data  area  describing  CNEWS  processes 
consists  of  all  courses  conducted  in  EW  operations, 
maintenance,  and  technology.  There  are  four  major 
kinds  of  operation  courses;  basic,  advanced,  watch¬ 
standing,  and  tactical  operations  as  shown  in  Table 
1.  Two  kinds  of  maintenance  courses  are  taught; 
preventive  and  corrective.  There  are  four  kinds 
of  technology  courses  taught;  digital,  3M  (Mainten¬ 
ance  and  Material  Management),  preventive,  and 
corrective.  In  addition  to  general  (conroon-core) 
courses  taught  in  operations  and  maintenance  train¬ 
ing,  a  number  of  equipment -specific  training 
courses  are  also  held  at  CNEWS.  The  four  major 
equipment-specific  areas  of  training  are:  AN/ 
ULR-1,  AN/WLR-8,  AN/SIQ-17,  and  AN/SLQ-32.  Table 
2  is  a  list  of  the  courses  taught  at  CNEWS  identi¬ 
fied  by  the  type  of  training,  the  course  title, 
its  length  in  weeks,  and  its  Course  Data  Processing 
(CDP)  number. 

The  Training  Track  Data  portion  of  the  EW 
student  flow  model  curriculum  track  process  de¬ 
scribes  significant  relationships  among  the  stu¬ 
dents  and  the  training  courses  they  take  at  CNEWS 
for  purposes  of  model  design  and  development. 

Table  1  also  lists  these  11  types  of  data. 

A  third  area  of  data  collected  to  aid  in  the 
development  of  the  model's  curriculum  track  pro¬ 


cesses  includes  CNEWS  training  resources  and  utili¬ 
zation  data.  For  each  course  taught  at  CNEWS,  nine 
major  areas  of  training  resource  information  and 
utilization  patterns  were  investigated  including: 

(1)  classrooms,  (2)  learning  centers  (including 
carrels  and  associated  audiovisual  media  such  as 
sound/slide  projectors,  8  urn  film,  cassette  record¬ 
ers,  videotape  players  and  student  responders),  (3) 
laboratories,  (4)  combination  classroom/laborator¬ 
ies,  (5)  training  devicas/simulators,  (6)  operation¬ 
al  equipment,  (7)  student/instructor  ratio,  (8)  stu¬ 
dent/equipment  ratio,  and  (9)  cross-utilization  of 
resources  (facilities  used  by  more  than  one  course; 
e.g.,  classroom  or  equipment). 

Current  CNEWS  Pipelines 

The  current  CNEWS  training  curriculum  for  en¬ 
listed  students,  which  includes  the  majority  of  the 
students  attending  the  school  consists  of  Basic 
Operations  (CDP  602A)  and  EW  Preventive  Maintenance 
Technology  (CDP602B)  courses,  followed  by  one  of  the 
equipment  operations  courses  as  appropriate  (AN/ 
WLR-1  (CDP  015A),  AN/SLQ-32  (CDP  016A,  AN/SLQ-17 
(CDP  107A) ,  or  AN/WLR-8  (CDP  018A)  followed  by  the 
appropriate  equipment-specific  preventive  mainten¬ 
ance  course.  All  students  then  take  Advanced  Oper¬ 
ations  (CDP  602D)  and  then  finish  their  training  at 
CNEWS  with  one  of  the  appropriate  equipment-specific 
tactical  operations  courses.  The  choice  of  what 
equipment-specific  course  students  take  is  deter¬ 
mined  by  their  subsequent  duty  assignments. 

In  the  current  CNEWS  curriculum,  all  courses 
are  group-paced  (lock-step)  except  for  the  Basic 
Operations  course  (CDP  602A).  The  Basic  Operations 
course  has  recently  been  developed  into  individual¬ 
ized  (self-paced)  programs  of  instruction.  The 
Basic  Operations  course  (CDP  602A)  is  significant 
since  most  all  categories  and  types  of  students, 
officer  and  enlisted,  normally  take  this  course. 
Figure  1  shows  a  more  detailed  diagram  of  the  CNEWS 
enlisted  student  flows.  It  is  based  upon  the  data 
collected  and  organized  in  Tables  1  and  2.  This 
figure  will  also  be  used  in  the  following  discussion 
to  show  what  changes  will  occur  with  the  implemen¬ 
tation  of  Device  10H1. 

Proposed  CNEWS  Pipelines  Using  Device  10H1 

The  EW  operator  curriculum  under  development 
for  Implementation  with  Device  10H1  will  be  indivi¬ 
dualized  or  self-paced  instruction.  The  major  dif¬ 
ferences  between  the  present  group-paced  (lock-step) 
operator  curriculum  and  the  individualized  curricu¬ 
lum  under  development  are:  student  self-pacing 
through  each  curriculum  module,  lesson  and  lesson 
topic.  Each  lesson  topic  is  completed  by  the  stu¬ 
dent  in  a  classroom,  a  learning  center  with  multi- 
media  learning  carrels  or  a  laboratory  using  one  of 
the  student  stations  of  Device  10H1.  The  learning 
carrels  consist  of  individual  study  booths  equipped 
with  various  combinations  of  sound/slide  equipment 
and/or  random-access,  interactive  videotape  equip¬ 
ment.  The  student  is  appropriated  support  with 
lesson  topic  narratives  and/or  progranmed  materials 
or  other  types  of  individualized  learning  materials. 

The  following  lists  the  CNEWS  student  tracks 
and  categories,  the  Device  10H1  operator  curricular 
phases  each  will  take,  and  the  current  CNEWS  course 
that  the  Device  10H1  operator  curriculum  phases  will 
replace  when  the  device  is  ready -for-training  and 
the  curriculum  has  been  developed  and  1$  in  place. 
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processes 


OUTPUTS 


INPUTS 

Type  Student* 
officers 


VQ  -  Pilot 
ASW-TACAIR 
EA6  PI  PREP 
VQ  Eval 
EA6  ECHO 
VAQ  33 
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.  Fleet  Returnee  (Operator/ 
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Fleet  Returnee  EL1NT 
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.  E1SJ 
.  CM  6*0 
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Curriculum  Track* 


Operations 


TRAINING  COURSE  PATA 
Maintenance 


Technology 


Basic  Preventive 

Advanced  Corrective 

.  Hatchstanding 
Tactical  * 


Digital 

W 

Preventive 

Corrective 


Equipment-Specific 
HUM  WlR-8  SlQ-17  SlQ-32 


TRAINING  TRACK  DATA 


RESOURCE  UTILIZATION  OAT* 


Sequence 

Type  Instruction 
Convening  Period/ Frequency 
Capacity 

Attrition/Setback  Ratios 

Course  Lengths 

Module 

lessons 

Tlnej-to-Traln 

Classrowi/Uboratory/ 

Trainer  Ratio 
COP  Number 


Learning  Center  {Including  Media) 
C 1 a s* room/L abora tory 
Laboratory 

Operational  Equipment 
Training  Dev  I ce/Si mulator 
Student/Instructor  Ratio 
Student/fquipeicnt  Ratio 
Cross-Utl Illation  by  Course* 


Multi -year  Requirements 
for  Graduates  by 
Student  Type 

Course  Operation  Reports 
Student  Time -To- Train 
Queues 


Table  1.  Consolidated  Navy  EW  School  Student  Flow  Simulation  Data  Base  Elements 


Table  2.  CNEIIS  Courses  by  CDP  Number,  Title,  and  Length 


Typer  Training 

Basic  Operation* 


Maintenance  and 
Technology 


6474 

6475 

9795 

9797 

9798 

9799 
9928 


COP  Nuaber* 
602A 


Cl* 
01 SA 
01 6A 
01 7A 
01  BA 
602C 
0158 
0168 
0178 
0188 
C2* 
C3* 
01  SO 
0160 
0170 
0180 
41 W 
0)6f 
01 7€ 
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604C 


Advanced  Operation*  6020 
01SC 
01 6C 
01 7C 
01 8C 
3117 


Officer  EH  Course* 


Title 


VQ-P1 lot  Navigator  EW  Course  (VQ  PIRER) 
Aviation  EW  Officer*  Course  (EV.0)  (ASW/ 
TACAIRKPMEP) 

CA-66  fleet  Replacement  Pilot  &  Course 
Fleet  Air  Reconnaissance  EW  Evaluator  Course 
EA-6B  Fleet  Replacement  NFO  ECVO  Course 
VAQ-33  NFO  fleet  EW  Suoport  Course 
EA-6  Marine  Aviation  EW  Course 

ENLISTED  EW  TECHNICIAN 
Course  Title 

EW  Technician  Class  A  School 
Sasic  EW  Operations 

Preventive  Maintenance  Technology 
AN/WIR-l  Operator-Equipment  Operations 
AN/SLQ-32  Opera tor-Equipment  Operations 
AN/SLQ-17  Operator-Equipment  Operations 
AN/WIR-8  Operator-Equipment  Ooeratlor.s 
3M  System/Test  Equipment  Training 
AN/WLR-1  Operator-Preventive  Maint. 
AN/SLQ-32  Operator-Preventive  faint. 
AN/SLC-17  Opera  tor- Preventive  faint. 
AN/Wi.ft-8  Operator-Preventive  Maint. 
Corrective  Maintenance  Technology 
Olgftal  Technology 
AN/WLR-1  ESM  System  Maintenance 
AN/SLQ-32V2  ESM  Mainten#nct-Ar</vrK-19 
AN/SLQ-17  Malntenance-AN/UTK-20 
AN/WLR-8  Maintenance 
AN/ULQ-6  Niintenanca 
AN/SL0-32V2  ESM  Maintenance-System 
AN/StO-17  Mafntrnanre 
AN/SL0-32V3  CCM  Maintenance 
Subemrlne  Electronics  Technician  EW 
Technology-Coerwn. /Radar  Theory 
Advanced  EW  Operations 
AN/WLR-1  Operator-Tactical  Ooerations 
AN/SLQ-32  Operator-Tactical  Operations 
AN/SLQ-17  Operator-Tactical  Operation* 
AN/WIR-8  Operator- Tactical  Ooerations 
Cryptologic  Tecnnfclan-Fleld  Type  </ 

Class  A  (CLINT  Operator) 


tengtn/ 

Weeks 


6 

19 

3 

3 

3 

2 

2 

2 

2 

12 


•C1.C2.C3  -  Multiple  CDRs  used  for  these  three  courses  to  Identify  various  categories  of  student*  attending. 
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C3  Digital  Technology 
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Technician  EW  Techno 1 ocy 
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Figure  1.  CNEWS  Student  Flow  by  Tracks  and  Courses  (Part  A) 


1-4 


These  are  shown  in  Figure  1 
shadowed. 

Current  CNEWS  Course 

COP  602A  Basic  Operations 

CDP  6020  Advanced  Operations 

CDP  01 5C  AN/WLR-1 

Tactical  Operations 
CDP  01 6C  AN/SLQ-32 

Tactical  Operations 
COP  01 7C  AN/SLQ-1 7 

Tactical  Operations 
CDP  01 8C  AN/SLQ  17 

Tactical  Operations 


with  the  course  boxes 


Replaced  By 
Device  10H1  Phase  1 
Device  10H1  Phase  11 


Device  10H1  Phase  III 


is  complete. 

When  the  student  reaches  phase  III  (watch¬ 
standing  and  tactical  operations)  in  Device  10H1 , 
his  training  will  become  more  equipment-  or  suite- 
oriented  and  will  consist  of  two  parts.  Part  A  of 
phase  III  consists  of  exercises  to  orient  the  stu¬ 
dent  to  the  type  of  equipment  he  will  operate  in 
his  duty  assignment.  Part  B  of  phase  III  consists 
of  a  series  of  exercises  or  lesson  topics  in  the 
form  of  mission  scenarios  using  the  equipment  suite 
he  used  in  Part  A  of  phase  III.  Student  flow 
through  phase  III  will  consist  of  a  series  of  hours 
of  training  in  Device  10H1  in  parts  A  and  B  on  the 
respective  EW  equipment  suite  modules  and  lesson 
topics  appropriate  for  bis  military  specialty  and/ 
or  next  duty  assignment. 


In  phase  I  (basic  operations),  students  will 
complete  a  series  of  academic  or  off-line  (non-lOHl) 
modules  in  learning  centers  and  then  receive  a 
series  of  modules  and  lesson  topics  using  Device 
10H1  to  complete  that  portion  of  the  curriculum. 

The  basic  operations  phase  of  the  individualized 
curriculum  using  Device  10H1  is  planned  to  be 
common-core;  i.e.,  each  category  of  student,  offi¬ 
cer  and  enlisted,  will  take  the  phase  I  courses. 


III.  SIMULATION  MODEL  DESIGN  ELEMENTS 

The  purpose  of  this  section  is  to  describe 
the  structure  of  the  EW  student  flow  simulation 
model  and  its  design  elements.  The  first  part  of 
the  section  discusses  the  level  of  detail  and  the 
design  alternatives  to  be  included  in  the  model, 
followed  by  a  description  of  SLAM  modeling  concept 
and  its  operational  characteristics. 


Phase  II  (advanced  operations)  of  the  Device 
10H1  curriculum  which  will  consist  of  a  series  of 
six  modules  and  a  number  of  lesson  topics  with  the 
number  of  hours  each  student  is  estimated  to  spend 
in  the  learning  centers  and  Oevice  10H1.  Unlike 
the  phase  I  student  traffic  flow,  the  student's 
path  through  phase  II  will  consist  of  first  a  series 
of  sessions  in  the  learning  center  for  each  module 
and  then  a  number  of  hours  of  training  on  Device 
10H1.  As  a  student  completes  each  module  In  Device 
10H1 ,  he  will  proceed  to  the  next  assigned  module 
taking  a  series  of  lesson  topics  in  the  learning 
center  followed  by  an  appropriate  number  of  hours 
or  lesson  topics  in  Device  10H1  until  that  module 


Level  of  Details  to  be  Obtained  From  the  Model 

The  objective  of  the  model  is  to  account  for 
each  different  type  of  student  in  the  various  pipe¬ 
lines,  whether  they  be  In  group-paced  or  individu¬ 
alized  training  courses,  and  gather  statistical 
data  on  each  student  in  order  to  determine  the 
CNEWS  resource  utilization  patterns,  as  well  as  to 
identify  likely  problem  areas  such  as  student 
queues  or  excessive  waiting  times.  In  particular, 
the  student  flow  simulation  model  will  be  designed 
to  produce  the  following  types  of  information. 

1.  The  maximum,  minimum,  and  mean  expected 
times-to-train  for  each  type  student 


pipeline  in  the  CNEWS  curriculum  as  de¬ 
fined  in  section  II. 

2.  Identification  and  definition  of  class¬ 
room/  learning  center/training  device  on 
equipment  resource  demand  situations  for 
the  courses  in  the  CNEWS  curriculum  over 
a  multi-year  planning  horizon. 

3.  Identification  and  definition  of  training 
resource/facility  requirements  to  achieve 
a  smooth  flow  of  students  through  the 
CNEWS  curriculum  including  group-paced 
and  individualized  courses  of  instruction 
in  terms  of  existing  school  capacity  and 
planned  capacity.  Major  variables  to  be 
considered  include: 

a.  the  number  and  types  of  students  and 
their  arrival  patterns, 

b.  course  convening  frequency, 

c.  instructional  materials,  media, 
training  devices,  and/or  operational 
equipment. 

4.  Determination  of  the  impact  to  the  school 
in  terms  of: 

a.  changes  in  student  input  distribution 
and  on-board  arrival  times, 

b.  the  addition  of  new  instructional 
resources  such  as  audio-visual  equip¬ 
ment,  learning  carrels,  or  training 
devices  such  as  Device  10H1, 

c.  the  incremental  conversion  of  courses 
in  a  curriculum  from  group-paced  to 
individualized  training  and  in  some 
cases  establishment  of  comnon-core 
training. 

Description  of  Design  Alternatives 

The  model  will  be  developed  in  seven  incre¬ 
ments  In  order  to  be  able  to  determine  the  impacts 
of  implementing  the  three  phases  of  the  Device 
10H1  curriculum.  The  model  will  be  developed  in 
the  following  increments: 

1.  CNEWS  Current  Operator  and  Maintenance 
Curriculum  (No  Device  10H1), 

2.  CNEWS  Current  Curriculum  With  Device  10H1 
Phase  I  (Basic  Operations)  Implemented 
and  Replacing  Existing  Basic  Operations 
Course, 

3.  CNEWS  Current  Curriculum  With  Device  10H1 
Phase  II  (Advanced  Operations)  Implemented 
and  Replacing  Existing  Advanced  Operations 
Courses, 

4.  CNEWS  Current  Curriculum  With  Device  10H1 
Phase  III  (Watchstanding-Tactical  Opera¬ 
tions)  Implemented  and  Replacing  Existing 
Tactical  Operations  Courses, 

5.  CNEWS  Current  Curriculum  With  Device  10H1 
Phases  I  and  II  Implemented  and  Replacing 
Existing  Basic  and  Advanced  Operations 
Courses, 

6.  CNEWS  Current  Curriculum  With  Device  10H1 
Phases  I  and  III  Implemented  and  Replac¬ 
ing  Existing  Basic  and  Tactical  Operation 
Courses , 

7.  CNEWS  Current  Curriculum  With  Device  10H1 
Phases  II  and  III  Implemented  and  Replac¬ 
ing  Existing  Advanced  and  Tactical  Opera¬ 
tions  Courses, 

8.  CNEWS  Current  Curriculum  With  Device  10H1 
Phases  I,  II,  and  III  Implemented  and 
Replacing  Existing  Basic,  Advanced,  and 
Tactical  Operations  Courses. 


Modeling  With  SLAM 

After  the  specific  model-building  process  had 
been  defined  and  the  structure  of  the  problem  spe¬ 
cified  as  discussed,  the  next  step  was  to  select 
an  appropriate  simulation  language  with  which  to 
model  the  decision  problem. 

SLAM  (2)  was  a  simulation  language  selected 
for  modeling  the  system.  The  language  allows  sys¬ 
tems  to  be  viewed  from  a  process,  event  and  state 
variable  standpoint.  The  authors  of  SLAM  claim 
that  the  language  combines  these  modeling  philoso¬ 
phies  in  a  unified  systems  modeling  framework. 

In  SLAM,  a  discrete  change  system  can  be  mo¬ 
deled  within  an  event  orientation,  process  orienta¬ 
tion  or  both.  Continuous  change  systems  can  be 
modeled  using  either  differential  or  difference 
equations.  Combined  discrete-continuous  change 
systems  can  be  modeled  by  combining  the  event  and/ 
or  process  orientation  with  the  continuous  orien¬ 
tation.  In  addition,  SLAM  incorporates  a  number  of 
features  which  correspond  to  the  activity  scanning 
orientation. 

Figure  2  presents  an  overview  of  the  compo¬ 
nents  involved  in  a  SLAM  systems  analysis.  Using 
the  SLAM  symbol  set  provided  in  (2),  the  analyst 
brings  to  the  systems  analysis  a  knowledge  of  the 
system  and  the  scenarios  to  be  evaluated.  In  our 
modeling  the  process  orientation  of  SLAM  is  most 
important.  The  process  orientation  of  SLAM  employs 
a  network  structure  comprised  of  specialized  sym- 
bol s  caTTed  nodes  and  branches  in  a  manner  similar 
to  Q-GERT.  These  symbols  model  elements  in  a  pro¬ 
cess  such  as  queues,  servers  and  decision  points. 
Based  on  the  scenarios  and  the  purpose  for  the 
mode  1  building  the  modeling  task  consists  of  com¬ 
bining  these  symbols  into  a  network  model  which 
pictorial ly  represents  the  system  of  interest.  In 
short,  a  network  is  a  pictorial  representation  of 
a  process.  The  entities  in  the  system  (such  as 
people  and  items)  fow  through  the  network  model. 

The  pictorial  representation  of  the  system  is 
transcribed  by  the  modeler  into  an  equivalent 
statement  model  for  input  to  the  SLAM  processor. 

After  the  data  base  file  has  been  created  and 
updated,  then  the  user  brings  the  SLAM  model  on¬ 
line  and  links  the  data  base  file  to  the  SLAM  model. 
Upon  completion  of  this  linkage  operation,  the 
user  then  calls  the  SLAM  processor  to  Interpret  die 
SLAM  source  program  in  the  FORTRAN  compiler  and  to 
execute  the  program.  Upon  completion  of  the  simu¬ 
lation,  a  standard  SLAM  sunmary  report  containing 
all  the  important  statistical  observations  is  auto¬ 
matically  provided.  The  output  format  designed  In 
a  SLAM  s unwary  report,  however,  is  general  in  na¬ 
ture.  It  will  be  necessary  to  redesign  the  output 
format  tailored  to  the  user's  easy-to-read  format. 
Further,  a  statistical  output  analysis  routine  will 
be  created  to  validate  any  statistical  significance 
of  observations  made  in  the  simulation. 

IV.  EW  STUDENT  FLOW  SIMULATION  MODEL  FEATURES 

This  section  describes  the  operational  fea¬ 
tures  of  the  EW  student  flow  simulation  model . 

The  first  part  of  this  section  presents  data  files 
created  to  run  the  simulation  model.  The  second 
part  of  this  section  discusses  typical  statistical 
output  features  from  the  model . 
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Figure  2.  Components  of  SLAM  Model 
Description  of  Model  Data  Base 


The  user  first  Interfaces  with  the  Course 
Data  Base  module  which  creates  and  maintains  the 
course  statistics  described  In  the  previous  sec¬ 
tions.  The  Course  Data  Base  module  consists  of  the 
five  subfile  elements  -  loHl  Status  profile.  In¬ 
structor  profile,  Facility  profile.  Course  Descrip¬ 
tion  profile,  and  Input  Student  Population  profile. 
That  Is,  the  Course  Data  Base  Is  a  collection  of 
training  course  records  which  provide  data  required 
for  operation  of  the  simulation  model.  The  speci¬ 
fic  data  elements  required  In  each  submodule  are 
Illustrated  In  Figure  3;  a  further  explanation 


Figure  3.  File  Elements  of  Course  Data  Base 


1.  Device  10H1  Profile.  This  file  basically 
contains  data  elements  which descri be  the  transi¬ 
tion  phase  from  group-paced  courses  In  the  CNEWS 
curriculum  to  individualized  training  courses  using 
Device  10H1.  Eight  configurations  of  the  transi¬ 
tion  are  included  In  the  profile: 

■  Device  10H1  Is  not  used. 

•  Device  10H1  is  used  for  Phase  I  (Basic  Op¬ 

erations)  only. 

•  Device  10H1  is  used  for  Phase  II  (Advanced 

Operations)  only. 

■  Device  10H1  is  used  for  Phase  III  (Tactical 

Operations  and  Watchstanding  Exercises). 

•  Device  10H1  is  used  for  both  Phases  I  and 

II. 

•  Device  10H1  Is  used  for  both  Phases  I  and 

III. 

■  Device  10H1  Is  used  for  both  Phase  II  and 

III. 

•  Device  10H1  is  used  for  both  Phases  I,  II, 

and  III. 

By  identifying  the  use  of  Device  10H1  with  an 
appropriate  function  code,  the  student  flow  simula¬ 
tion  model  will  automatically  assume  the  proper 
transition  phase  from  group-paced  courses  to  indi¬ 
vidualized  training  courses.  Therefore,  the  user 
does  not  have  to  look  into  the  simulation  model  to 
verify  any  changes  to  be  made  every  time  he  runs  a 
different  set  of  10H1  transition  phases. 

2.  Instructor  Profile.  The  instructor  pro¬ 
file  file  Is  designed  to  store  and  update  the  num¬ 
ber  and  types  of  instructors  available  to  each 
course  at  the  time  the  simulation  run  is  made.  As 
the  Course  Data  Processing  (CDP)  code  is  a  unique 
identifier  in  the  Navy,  the  code  Is  used  in  the 
model  as  the  key  for  instructor  identification. 

The  number  and  type  of  Instructors  available  to 
each  course  are  thus  entered  into  the  file  accord¬ 
ing  to  CDP  code.  Whenever  more  than  one  CDP  code 
represents  the  same  course  utilization,  a  new  CDP 
code  is  created  so  that  cross  utilization  of  the 
same  course  by  different  types  of  students  can  be 
Identified. 

3.  Facility  Status  Profile.  The  facility 
status  file  will  contain  only  data  elements  such 
as  various  training  devices,  classrooms  and  lab 
equipment  which  describe  the  current  or  anticipated 
status  of  various  instructional  resources  available 
for  individualized  (self-paced)  training  courses. 
Since  any  changes  in  the  use  of  instructional  re¬ 
sources  used  for  group-paced  courses  affect  only 
class  size  and/or  course  length  variables,  there 

Is  no  need  to  consider  these  factors  explicitly  in 
this  simulation. 
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4.  Course  Description  Profile.  The  course 
description  file  contains  the  various  course  sta¬ 
tistics  required  for  use  during  the  simulation. 
Specific  input  course  data  elements  to  be  keyed  in 
the  file  according  to  CDP  code  will  include:  the 
course  length,  the  class  size  (for  group-paced 
courses  only),  the  course  convening  frequency 
(grouped-paced  courses  only),  and  the  student 
course  attrition  and  setback  rates. 

for  individual ized  courses,  the  course  lengths 
will  depend  upon  the  individual  student's  perfor¬ 
mance.  Thus,  estimates  of  the  minimum,  maximum, 
and  average  times-to-complete  for  each  course  by 
the  students  will  be  required  aS  inputs  to  the 
model  instead  of  a  single  course  length  figure,  as 
with  group-paced  courses. 

5.  Input  Student  Distribution  Profile.  The 
input  student  distribution  profile  maintains  data 
elements  regarding  planned  student  inputs  for  each 
category  of  student.  The  projected  number  and  on¬ 
board  time  of  arrival  at  the  school  for  each  cate¬ 
gory  of  student  flowing  through  the  CNEWS  curricu¬ 
lum  will  be  used  in  the  simulation  model. 


Description  of  the  Model 's  Outputs 


The  model's  outputs  derived  from  its  Inputs 
and  process  interactions  are  provided  in  the  fol¬ 
lowing  case  study  to  demonstrate  the  effectiveness 
of  using  SLAM  as  a  simulation  language.  The  spe¬ 
cific  assumptions  being  made  in  the  case  study 
model  are: 

a.  The  design  alternative  assummed  was  the 
case  of  CNEWS  current  curriculum  with 
Device  10H1  used  for  Phase  I  (CDP  602A) 
only. 

b.  Seven  different  types  of  enlisted  EW  stu¬ 
dents  were  considered  with  27  different 
individual  courses.  The  basic  structure 
of  the  model  was  based  upon  the  student 
flow  model  described  in  Figure  1. 

c.  When  courses  are  convened  on  the  basis  of 
group-paced  Instruction,  provisions  were 
made  to  open  up  multiple  sessions  to  re¬ 
duce  the  students'  waiting  times.  A 
policy  was  set  up  to  convene  a  course  on 
a  regular  basis.  The  minimum  and  maximum 
class  sizes  were  established  for  conven¬ 
ing  each  course. 

d.  A  student  setback  and  attrition  problem 
associated  with  each  course  was  incorpor¬ 
ated  in  the  model.  The  detailed  input 
data  descriptions  are  given  in  (1). 


Assumming  400-student  annual  inputs  to  the 
model  over  the  two-year  period,  the  following  spe¬ 
cific  outputs  can  be  obtained. 


Number  of  Students  Graduated  Per  Course  Per 
Period  of  Run 


b.  Standard  deviation  of  number  of  students 
per  class. 

c.  Coefficient  of  variation  of  number  of 
students  per  class. 

d.  Minimum  number  of  students  in  a  class  over 
a  period  of  simulation. 

e.  Maximum  number  of  students  in  a  class  over 
a  period  of  simulation. 

f.  Number  of  classes  for  each  type  of  course 
that  has  met  over  period. 

g.  Completion  time  for  each  type  of  student 
per  course  over  period. 

h.  Total  number  of  students  who  dropped  from 
courses  over  period  of  simulation. 

It  is  seen  from  Table  3  that  the  average  time 
in  the  training  system  for  the  USN  6Y0  was  64.67 
weeks  with  a  standard  deviation  of  9.394  weeks  and 
times  ranged  from  41.16  weeks  and  86.07  weeks.  The 
distribution  for  time  in  the  system  is  depicted  by 
the  histogram  generated  by  SLAM  and  shown  in  Figure 
4.  There  were  a  total  of  49  observations  which 
means  that  forty-nine  students  of  USN  6Y0  completed 
the  curriculum  over  the  two-year  training  period. 

Another  statistic  of  interest  is  the  class 
size  of  each  course  and  number  of  class  convening 
frequency.  For  this  example,  NUM  STUD  S  C602  in 
Table  3  stands  for  the  class  size  of  CDP  602C  when 
it  was  offered  on  the  basis  of  single  session  where¬ 
as  NUM  STUD  D  C602  does  when  it  was  offered  on  the 
basis  of  parallel  double  sessions.  The  average 
class  size  for  a  single  session  was  8.5  students 
with  a  standard  deviation  of  .7071  students.  The 
size  of  class  was  ranged  from  8  to  10  students. 

There  were  10  single  session  offerings  over  the  two- 
year  period. 

Courses  016F,  017D,  017E,  and  018D  were  not 
convened  because  there  were  not  enough  students  to 
take  those  courses  during  the  simulation  period. 
These  facts  are  Identified  in  the  simulation  output 
by  "NO  VALUES  RECORDED". 


2.  Course  Waiting  Times 


The  second  category  of  statistics  for  this 
example  is  the  file  statistics.  Table  4  Illustrates 
course  waiting  statistics  collected  from  the  course 
files.  The  specific  file  numbers  in  column  1  corre¬ 
spond  to  course  numbers  assigned  in  the  network 
model.  The  rest  of  the  columns  represent  the  fol¬ 
lowing  pieces  of  information  collected  over  time  of 
model  run. 

a.  The  average  number  of  students  in  the  file 
over  time. 

b.  The  standard  deviation  of  the  number  of 
entitles  in  the  file  over  time. 

c.  The  maximum  number  of  students  in  the  file 
at  any  one  time. 

d.  The  current  number  of  students  in  the  file. 

e.  The  average  waiting  time  in  the  file. 


Table  4  shows  the  first  category  of  statistics 
for  variables  based  upon  observations.  For  this 
model,  these  statistics  were  collected  by  the  net¬ 
work  model  and  include  the  interval  statistics  for 
average  class  size  for  each  course  and  training 
time  for  each  type  of  student.  The  first  column  in 
Table  3  represents  designations  of  classes  and  stu¬ 
dent  training  tracks.  In  particular  the  following 
information  can  be  obtained  by  reading  down  the 
columns  and  rows  from  the  table. 

a.  Average  number  of  students  per  class. 


For  this  example.  File  No.  1  represents  the 
number  of  students  waiting  for  course  convening  of 
CDP  602A.  Thus,  the  average  number  of  students 
waiting  for  CDP  602A  was  6.6343,  with  a  standard 
deviation  of  4.8032  students,  a  maximum  of  20  stu¬ 
dents  waited  and  at  the  end  of  simulation  there 
were  17  students  in  the  queue.  Finally,  the  average 
waiting  time  by  the  students  was  0.8917  weeks. 


203 


Table  3.  Statistics  for  Variables  Based  on  Observation 
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Figure  4.  A  Sample  Histogram  for  Training  Time  Variable  of  the  USN  GYO. 


204 


3.  Course  Utilization  Statistics 

The  third  category  of  statistics  for  this 
example  is  statistics  on  regular  activities.  The 
activity  index  in  Table  5  presents  various  activi¬ 
ties  associated  with  training  course  curriculum. 

In  particular,  the  following  statistics  can  be  ob¬ 
served  from  the  table  over  the  two-year  simulation 
period. 

a.  The  average  number  of  students  undertak¬ 
ing  the  activity. 

b.  The  standard  deviation  of  the  number  of 
students  undertaking  the  activity  over 
time. 

c.  The  maximum  number  of  students  undertak¬ 
ing  the  activity  at  any  one  time. 

d.  The  number  of  students  which  have  comple¬ 
ted  the  activity. 

For  this  example,  the  activity  number  1  in  the 
first  column  in  Table  5  represents  the  course  util¬ 
ization  statistics  for  COP  602A.  The  results  show 
that  542  students  completed  the  course  during  the 
two-year  time  units  of  simulation,  and  that  there 
was  an  average  of  21.6248  students  in  transit  be¬ 
tween  the  602A  and  the  Cl  (see  Figure  1).  The  max¬ 
imum  number  of  students  undertaking  the  course  at 
any  one  time  was  36.  There  are  currently  24  stu¬ 
dents  taking  the  course  at  the  end  of  simulation 
run. 

4.  Resource  Activity  By  Type  of  Training  Facility 
(Device  lOHll- 

The  last  category  of  statistics  for  this  ex¬ 
ample  is  statistics  on  service  activities.  Recall 
that  Device  10H1  was  used  to  convert  CDP  602A  into 
a  Self-paced  instructional  system.  Thus,  Table  6 
summarizes  resource  utilization  statistics  for  De¬ 
vice  10H1 .  For  example,  the  following  pieces  of 
information  can  be  obtained. 

a.  The  current  capacity  of  the  resource 
(Device  10H1). 

b.  The  average  utilization  of  the  resource 
over  time. 


Table  5.  Course  Utilization  Statistics 
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c.  The  standard  deviation  of  the  resource 
utilization  over  time. 

d.  The  maximum  number  of  units  of  resource 
utilized  at  any  one  time. 

e.  The  current  number  of  units  of  resource 
utilized. 

In  our  case  example,  the  results  show  that  on  the 
average  23.4185  of  the  60  units  of  Device  10H1 
were  utilized  and  that  40  units  of  Device  10H1  were 
busy  at  any  one  time  over  the  simulation  period. 
This  implies  that  there  be  more  than  enough  units 
of  Device  10H1  assigned  to  the  CDP  602A.  However, 
when  courses  602D,  015C,  016C,  017C,  and  018C  are 
eventually  converted  into  self-paced  courses  re¬ 
quiring  Device  10H1  service,  a  better  utilization 
of  Device  10H1  is  expected. 

Table  6.  Device  10H1  Utilization  Statistic _ 
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V.  POTENTIAL  USES  OF  THE  MODEL 

This  section  discusses  some  potential  future 
uses  the  student  flow  simulation  model  has  after  it 
is  developed  and  evaluated. 
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Earlier  Planning  Possible 

CNEUS  planners  use  of  a  student  flow  model 
should  increase  their  training  effectiveness  by 
allowing  them  to  consider  and  test  a  variety  of 
course  combinations  before  a  course  or  curriculum 
configuration  is  chosen  for  final  development  and 
implementation.  The  use  of  a  student  flow  model 
should  allow  course  planning  to  take  place  earlier 
and  provide  a  means  for  more  effective  planning  dia¬ 
log  among  individual  course  developers,  course  oper¬ 
ations  personnel,  and  CNEWS  planners.  The  model 
will  allow  course  and  curriculum  planning  to  include 
a  larger  number  of  curriculum  design  options  to  be 
considered.  It  may  be  possible  to  adapt  future 
versions  of  the  model  to  CNEWS  management  informa¬ 
tion  system  (MIS)  efforts,  and  selected  cost  fea¬ 
tures  could  be  developed  for  the  model  that  could 
reflect  the  Navy's  training  resource  acquisition 
and  management  policies  and  directives  at  the  CNEUS 
operational  level. 

A  significant  potential  advantage  of  the  stu¬ 
dent  flow  simulation  model  for  CNEWS  planners  is  in 
the  development  and  implementation  of  the  individu¬ 
alized  EW  operator  curriculum  and  the  introduction 
of  the  multi-stationed,  generalized  EW  operator 
trainer.  Device  10H1.  This  will  include  the  ability 
to  examine  quickly  the  likely  impact  on  the  curri¬ 
culum  of  adopting  new  training  technology  and  meth¬ 
ods  for  courses  of  instruction  being  planned  to  meet 
new  training  requirements  as  they  arise. 

The  selection  of  data  to  use  as  inputs  to  the 
development  of  the  EW  student  flow  model  was  made 
with  these  types  of  future  capabilities  in  mind. 

The  model  will  allow:  (1)  the  interactive  examina¬ 
tion  of  the  essential  elements  of  curriculum  plan¬ 
ning  in  terms  of  a  range  of  choices,  (2)  the  identi¬ 
fication  of  the  advantages  and/or  disadvantages  of 
such  choices  as  they  are  likely  to  affect  student 
flow  and  facility/resource  operations,  and  (3) 
planning  checks  for  internal  scheduling  conflicts 
as  curriculum  choices  are  made. 

Likely  Impacts  of  Planned  Alternatives 

The  model  should  be  of  value  to  EW  school 
curriculum  planners  in  considering  the  impact  of 
alternatives  to  meet  changes  in  training  require¬ 
ments  by  providing  insights  into  the  likely  impacts 
of  such  alternatives  prior  to  having  to  make  large 
and  often  long-term  investments  in  time  and  re¬ 
sources.  Such  insights  might  include,  for  example, 
"optimizing"  the  change  over  from  lockstep  to  indi¬ 
vidualized  portions  of  the  curriculum.  The  "opti¬ 
mization"  might  take  place  in  terms  of  altering 
course  convening  frequencies  to  minimize  competition 
for  available  resources,  or  the  identification  of 
potential  course  conflicts  so  that  alternatives 
could  be  considered  such  as  course  restructuring  or 
the  use  of  double  shift  operations  for  a  limited 
time. 

Since  the  EW  school  will  be  shifting  from 
lockstep  courses  to  individualized  course  in  the  EW 
operator  curriculum  in  a  phased  program  over  several 
years,  the  student  flow  model  should  provide  in¬ 
sights  into  the  likely  Impact  of  such  an  Incremental 
program  and  the  likely  operational  problems  to  be 
encountered  when  students  exist  from  a  lockstep 
course  and  enter  an  individualized  course,  or  vice 
versa.  This  condition  will  occur  during  the  next 
few  years  since  the  maintenance  courses  will  remain 


lockstep  for  the  near  future  and  the  operator  train¬ 
ing  courses  will  be  incrementally  converted  to  in¬ 
dividualized  courses.  The  model  will  provide  in¬ 
sights  into  the  probable  Impacts  of  such  changes 
and  will  provide  forecasts  of  the  likely  effects  of 
establishing  and  operating  dual  pipelines  for  a 
period  of  time  during  such  a  changeover. 
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ABSTRACT 

Instructional  Systems  Development  (ISO)  In  the  military  has  traditionally  either  been 
performed  In-house  or  by  an  Independent  contractor.  Historically,  prime  airframe  manu¬ 
facturers  have  neither  had  the  Interest  nor  the  technical  capabilities  to  provide  this 
service.  Based  strictly  upon  the  technical  requirements  for  conducting  ISD  for  an  emerging 
weapon  system,  however,  the  airframe  manufacturer  Is  In  a  unique  position  to  concurrently 
design  and  develop  training  In  parallel  with  the  aircraft.  He  has  ready  access  to  the 
technical  data,  engineering  specification  and  design  Inputs  much  earlier  than  any  outside 
agency.  His  ability  to  collect  performance  data  and  determine  system  configuration  can 
expedite  the  development  of  training  devices  specifications  thus  Insuring  that  devices  are 
delivered  In  time  to  begin  Initial  aircrew  training.  Additionally,  training  requirements 
can  be  considered  early  enough  to  be  integrated  Into  the  design  process  and  actually  Impact 
the  final  system  configuration.  With  the  emergence  of  a  "total  systems  approach"  mandated 
by  0MB  Circular  A109  and  DOD  5000.1  and  .2,  this  ISD  capability  Is  being  developed  by  the 
airframe  Industry  and  should  be  considered  as  an  Integral  part  of  the  prime's  responsi¬ 


bility  during  Full  Scale  Development. 

INTRODUCTION 

Traditionally,  the  role  of  the  airframe 
manufacturer  In  Instructional  systems  development 
has  been  limited  to  support  of  the  process  through 
providing  technical  data  and  documentation  on  the 
system  configuration  and  performance.  As  part  of 
contractual  obligation  during  Full-Scale  Engineer¬ 
ing  Development  (FSED),  he  is  typically  required 
to  develop  a  hunan  factors  task  analysis  for 
system  operators  and  logistic  support  analysis  for 
maintenance  functions.  In  addition,  he  Is 
required  to  develop  Type  1  training  and  identify 
training  equipment  needed  to  support  Operational 
Testing  and  Evaluation  (OTSE)  for  the  Initial 
cadre  of  system  operators  and  malntalners.  In 
this  role,  little  of  what  he  does  Is  Immediately 
applicable  to  the  formal  Instructional  Systems 
Development  (ISD)  process  required  to  analyze, 
design  and  develop  aircrew  and  maintenance  train¬ 
ing  programs  for  operational  personnel.  Recent 
modifications  to  the  system  acquisition  process 
(e.g.,  OW  A- 109  and  DOD  5000.1  and  .2),  which 
dictate  a  total  systems  approach,  are  realigning 
these  traditional  roles  and  placing  the  prime  con¬ 
tractor  In  the  forefront  of  the  analysis  process 
to  Insure  that  training  and  personnel  requirements 
are  appropriately  addressed  from  the  outset  of 
system  conceptualization.  These  changes,  requir¬ 
ing  the  prime  contractor  to  be  heavily  Involved 
In  the  front-end  analysis,  provides  an  oppor¬ 
tunity  for  him  to  contribute  significantly  to  the 
design  and  development  of  training  early  In  the 
system  acquisition  process.  In  fact,  It  makes  him 
the  logical  candidate  to  perform  ISD  In  parallel 
with  the  design  and  development  of  the  air  vehicle. 

There  are  those,  of  course,  who  would  argue 
with  this  conclusion.  The  view  Is  held  by  many 
that  the  ISD  requirement  Is  best  satisfied  by 
anyone  but  the  prime  airframe  manufacturer.  This 
op1n1on~Ts  predominately  shared  by  those  whose 
main  product  line  Is  training  support  and  services, 
as  well  as  a  few  skeptical  military  customers. 

The  reasons  given  cite  the  opinion  that  most 


airframe  manufacturers  do  not  have  sufficient  tech¬ 
nical  expertise  nor  an  adequate  background  and/or 
Involvement  In  training  technology  research  and 
development  to  remain  current.  They,  also,  point 
out  that  a  prime  contractor's  main  responsibility 
Is  the  air  vehicle  and  that  everything  else  Is 
secondary;  hence,  manpower,  budgets,  and  schedules 
will  not  receive  sufficient  management  priority  In 
the  training  areas.  In  essence  they  maintain  there 
are  too  many  competing  elements  to  allow  training 
to  receive  the  attention  It  deserves.  This  may  be 
true  In  some  Instances  but  this  problem  Is  not 
unique  to  airframe  manufacturers.  The  lack  of  a 
proper  management  comnltment  to  training  Is  an 
Institutional  and  organizational  problem,  not  a 
technical  problem,  and  thus  can  be  solved  through 
enlivened  management  and  program  supervision 
supported  by  corporate  policy.  Those  who  maintain 
that  the  prime  contractor  should  not  perform  ISD, 
however,  will  also  be  the  first  to  admit  that  ISD 
cannot  be  performed  without  the  prime  contractor. 
Hence,  we  have  a  classic  dllenma  as  to  what  his 
proper  role  should  be? 

INSTRUCTIONAL  SYSTEMS  DEVELOPMENT  BY  THE  PRIME 

The  prime  normally  conducts  the  initial  task 
analysis  as  part  of  the  hunan  factors  design 
activities.  Rather  than  being  a  legitimate  task 
analysis  It  Is  typically  a  task  Inventory  or  list¬ 
ing  Instead  of  a  complete  analysis  of  the  tasks 
and  their  behavlorlal  requirements.  However,  If 
properly  conducted  and  performed  In  light  of  future 
ISD  requirements,  this  Initial  task  analysis  can  be 
expanded  to  Include  performance  conditions  and 
standards;  task  frequency,  criticality  and  diffi¬ 
culty;  task  Initiation  and  termination  cues;  crew 
and  crew  coordination  responsibilities;  and  system/ 
subsystem  Interfaces.  The  addition  of  this 
analytic  data  to  the  task  Inventory  can  then  be 
directly  applied  to  subsequent  ISD  activities  and 
the  development  of  functional  specifications  for 
training  devices  In  a  more  timely  and  logical 
manner.  In  the  early  stages  of  system  engineering 
and  design,  the  prime  airframe  manufacturer  Is  the 
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only  one  who  has  access  to  adequate  technical  data 
to  perform  such  a  detailed  analysis.  Thus,  he  can 
conduct  the  initial  ISD  step  (analysis  of  job  per¬ 
formance  requirements)  much  earlier  than  either 
independent  contractors  or  the  military  using  com¬ 
mand's  in-house  staff. 

More  critical  than  the  early  ISD  analysis 
step  is  the  ability  to  expedite  the  identification 
and  definition  of  training  device  requirements  so 
that  long  lead  time  items  required  to  develop 
training  equipment  can  be  adjusted  to  match  air¬ 
craft  delivery  schedules.  Historically,  the  ISD 
process,  which  strives  to  optimize  training 
resources,  is  inhibited  or  constrained  by  train¬ 
ing  equipment  which  does  not  adequately  address 
training  requirements  or  is  not  available  in  time 
to  be  integrated  into  the  training  program  at 
Reag,  for  Training  (RFT)  dates.  There  are  several 
factor:,  contributing  to  this  situation.  (1)  The 
ISD  proctss  is  not  initiated  early  enough  to 
impact  training  device  design,  (2)  Training 
devices  are  designed  by  engineers  with  insuffi¬ 
cient  data  concerning  training  needs,  and 
(3)  Training  device  manufacturers  must  rely  upon 
technical  data  from  the  airframe  contractor  which 
is  typically  not  timely  nor  sufficiently  detailed. 
All  three  of  these  factors  can  be  eliminated  or 
minimized  if  the  airframe  manufacturer,  as  the 
instructional  systems  designer  and  developer,  is 
also  responsible  for  the  detailed  functional  speci¬ 
fication  of  device  characteristics  and  performance. 
ISD  personnel  can  be  integrated  directly  into  the 
airframe  design  process  where  they  have  access  to 
current  technical  data  and  documentation  and  are 
able  to  influence  the  aircraft  design  in  areas 
that  could  effect  training  and  ultimately  system 
effectiveness . 

The  prime  contractor  designs  and  develops 
Type  1  training  as  part  of  his  FSED  contract. 
Normally,  this  training  does  not  follow  the  sys¬ 
tematic  and  orderly  process  of  ISD  and,  hence,  is 
not  adequate  in  content  or  format  for  formal  opera¬ 
tional  training.  If,  however,  the  airframe  manu¬ 
facturer  was  also  responsible  for  the  ISD  of  air¬ 
crew  and  maintenance  training  programs.  Type  1 
training  materials  and  equipment  could  be  designed 
and  developed  using  standard  ISD  methodology  so 
that  it  could  be  integrated  into  follow-on  train¬ 
ing  programs.  This  approach  would  not  only  be 
more  effective  and  efficient  in  terms  of  time, 
resources  and  costs  but  it  would  also  provide  an 
opportunity  to  validate  the  training  during  0T8E 
and  make  required  revisions  prior  to  Initial 
Operational  Capability  (IOC)  and/or  RFT.  In 
addition,  actual  training  equipment  required  for 
operational  training  could  be  prototyped  and 
tested  simultaneously  with  the  aircraft,  by  the 
Test  Forces  during  Type  1  training,  allowing  RFT 
certification  prior  to  starting  operational  train¬ 
ing.  Not  only  would  this  procedure  insure  that 


the  devices  met  the  training  requirements  but  it 
would  also  insure  that  the  full  suite  of  devices 
are  in  place,  providing  a  totally  integrated 
training  system,  for  formal  training  which  is 
something  that  has  not  been  possible  to  date. 

SUMMARY 

The  prime  manufacturer's  extensive  involve¬ 
ment  in  the  "front-end"  analysis, mandated  by  the 
new  procurement  procedures,  will  increase  his 
knowledge  and  understanding  of  operational  require¬ 
ments  needed  to  design  hardware  to  meet  mission 
needs.  His  primary  concern  must  be  with  a  total 
system  concept,  not  just  meeting  hardware  and 
software  specifications.  Personnel  and  training 
requirements  are  an  integral  part  of  the  total 
system  and,  hence,  must  be  integrated  into  the 
prime  contractor  design  and  development  process 
from  the  outset.  Thus,  airframe  manufacturers 
must  develop  the  technical  capabilities  to  deal 
with  these  expanded  areas  of  responsibility. 

Given  this  changing  set  of  conditions  for  doing 
business  with  the  military  customer  and  the  need 
to  broaden  technical  expertise  to  include  state- 
of-the-art  training  technology,  the  airframe 
manufacturer  is  now  in  a  position  to  assume  the 
role  of  an  instructional  system  designer  and 
developer  in  addition  to  being  the  designer  and 
developer  of  the  aircraft.  He  has  immediate 
access  to  all  the  prerequisite  technical  data  and 
documentation.  ISD  can  be  integrated  with  system 
engineering  and  delivery  schedules  for  the  air 
vehicle.  His  subject  matter  experts  (SME's)  will 
be  system  engineers  and  test  pilots  with  detailed 
knowledge  of  system  components  and  operations. 

He  has  access  to  SME's  from  the  outset,  hence 
facilitating  his  ability  to  complete  early  analysis 
of  system  requirements.  These  capabilities,  com¬ 
bined  with  the  requirements  for  a  total  systems 
approach,  logically  argues  that  the  system  and  all 
its  constituent  elements  should  be  derived  from  a 
cornnon  source— the  prime  contractor.  Otherwise, 
instead  of  a  totally  integrated  system,  the  results 
will  continue  to  be  a  series  of  subsystems,  each 
independently  optimized  for  its  own  unique  function 
resulting  in  suboptimization.  This  is  clearly 
not  the  goal  of  the  more  effective  and  efficient 
system  procurement  processes  being  developed.  The 
time  has  come  for  training  to  become  an  integral 
part  of  the  system  and  for  the  prime  contractor  to 
assume  the  role  and  responsibility  for  ISD  just  as 
with  any  other  element  of  the  systems.  In  an  age 
of  shrinking  budgets  and  escalating  costs,  this  is 
the  only  solution  which  makes  sense  either  in  terms 
of  system  effectiveness  or  efficiency.  The  air¬ 
frame  industry  is  prepared  to  meet  the  challenge 
for  total  system  responsibility  and  accountability, 
including  all  the  elements  of  personnel  and  train¬ 
ing  required  to  support  that  system. 
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OPTIMIZING  SIMULATOR-AIRCRAFT  TRAINING  MIXES* 
William  R.  Bickley 

U.S.  Army  Research  Institute  Field  Unit  at  Fort  Rucker 


ABSTRACT 

Under  the  assumptions  that  (1)  as  simulator  training  increases,  required  aircraft 
training  decreases  to  some  non-negative  minimum  and  that  (2)  at  any  point,  the  rate  at  which 
required  aircraft  training  decreases  is  a  fixed  proportion  of  the  difference  between  present 
required  training  and  the  minimum  required  training  achievable,  the  function  relating  x,  the 
simulation  training  received,  with  y,  the  subsequent  training  required  in  the  actual  aircraft 
to  attain  criterion,  will  be  of  the  form  y  =  ae"°x  +  c.  This  formulation  has  tremendous 
utility  in  allowing  the  training  analyst  to  calculate  the  most  cost  effective  mixes  of  simu¬ 
lator  and  aircraft  training.  This  approach  was  applied  in  the  U.S.  Army's  acceptance  tests 
of  the  AH-1  flight  simulator  (AH1FS).  Non-linear  regression  analysis  of  data  collected  on 
some  30  individual  maneuvers  indicates  the  methodology  is  viable.  A  straightforward  method¬ 
ology  for  incorporating  these  results  into  analysis  of  the  combined  cost  and  training 
effectiveness  of  the  AH1FS  and  similar  training  devices  is  presented. 


Implicit  in  the  acquisition  of  any  simulation 
training  system  is  the  assumption  that  training 
objectives  are  more  economically  attained  through 
a  mix  of  simulation  and  hands-on  training  than 
through  hands-on  training  alone.  This  was  the 
concept  guiding  the  U.S.  Army  when  in  1967  the 
basic  requirements  and  projections  for  a  Synthetic 
Flight  Training  System  (SFTS)  were  first  elabo¬ 
rated.  Faced  on  the  one  hand  by  rapidly  increas¬ 
ing  training  and  operating  costs  and  encouraged 
on  the  other  by  advances  being  made  in  training 
simulator  technology,  the  Army  embarked  on  a  long- 
range  SFTS  development  program,  developing  first 
the  UH-1  Iroquois  (Huey)  instrument  flight  simula¬ 
tor  and  then  the  CH-47  Chinook  visual  flight 
simulator.  The  latest  addition  to  the  SFTS 
program  is  the  AH-1  Cobra  visual  flight  and 
weapons  system  simulator  (AH1FS). 

This  paper  reports  in  detail  a  novel  method¬ 
ology  used  in  determining  the  training  effective¬ 
ness  of  the  AH1FS  as  a  training  medium  for  tran¬ 
sitioning  rated  rotary  wing  aviators  to  the  AH-1 
aircraft.  The  report  is  presented  in  three 
general  sections.  The  first  describes  the 
derivation  of  the  methodology;  the  second  presents 
the  results  of  applying  this  methodology  in  test¬ 
ing  the  AH1FS;  and  the  third  describes  how  the 
data  obtained  is  to  be  used  in  determining  opti¬ 
mal  mixes  of  simulator  and  aircraft  training. 

METHODOLOGY  FORMULATION 
Training  Effectiveness 

As  indicated  above,  the  motive  driving  train¬ 
ing  simulator  development  is  economy  in  training. 
The  economy  achieved  is,  of  course,  determined  by 
the  cost  and  the  effectiveness  of  a  unit  of  simu¬ 
lator  training  relative  to  the  cost  and  effective¬ 
ness  of  a  unit  of  (in  the  present  case)  aircraft 


♦The  views,  opinions,  and/or  findings  con¬ 
tained  in  this  report  are  those  of  the  author(s) 
and  should  not  be  construed  as  an  official  Depart¬ 
ment  of  the  Army  position,  policy,  or  decision, 
unless  so  designated  by  other  official  documen¬ 
tation. 


training.  With  past  simulators,  the  cost  differ¬ 
ential  between  simulator  and  aircraft  training  has 
been  so  great  that  a  marginally  effective  simula¬ 
tor  might  be  used  to  realize  overall  training 
savings.  However,  as  simulators  have  grown  more 
complex  and  expensive  to  operate,  the  differential 
has  shrunk  to  the  point  that  precise  quantitative 
determination  of  training  effectiveness  is  becom¬ 
ing  a  major  step  in  U.S.  Army  simulator  testing 
and  acceptance  procedures. 

Training  effectiveness  can  be  viewed  and 
defined  in  many  ways,  but  as  Roscoe  (1)  points 
out,  traditional  measures  of  effectiveness  fail  to 
take  into  consideration  costs  associated  with  pre¬ 
training  or  simulator  training.  Roscoe  has  pro¬ 
posed  the  cumulative  transfer  effectiveness  ratio 
(CTER)  as  an  alternative  with  more  utility  for  the 
training  psychologist.  The  CTER  is  defined  as 

CTER  =  (x0  -  y-j)  /  xi, 

where  Xj  is  training  received  in  a  simulator,  ^ 
is  training  required  in  the  aircraft  after  X.- 
simulator  training,  and  x.  is  training  that  would 
be  required  in  the  aircraft  were  no  simulator 
available.  The  f/f‘io  comr  r  training  savings  in 
the  aircraft  a,  ‘unction  «foount  of  simulator 
training;  a  t,'FK  of  .75  would  indicate  that  for 
some  xj  units  of  simulator  training,  each  unit  is 
equivalent  to  .75  unit  of  aircraft  training.  The 
CTER  has  great  utility  to  the  training  psychologist 
as  a  measure  of  training  effectiveness  and  was 
quite  successfully  employed  in  Holman's  (2)  eval¬ 
uation  of  the  CH-47  flight  simulator. 

But,  as  Roscoe  points  out,  the  CTER  is  not  a 
constant  but  is  very  much  a  decreasing  function  of 
the  value  of  x.j.  In  fact,  if  Xg  -  yj  ^  0,  it  can 
be  seen  that 


Thus,  it  is  the  case  that  each  empirically  estab¬ 
lished  CTER  will  be  valid  only  for  some  arbitrarily 
small  neighborhood  around  its  particular  x ^ . 
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From  the  training  psychologist's  point  of  view, 
an  ideal  measure  of  training  effectiveness  should 
convey  the  same  information  as  does  the  CTER  but 
should  also  allow  computation  of  training  effec¬ 
tiveness  for  all  x.j .  This  could  be  accomplished 
by  regressing  CTEK'  on  various  experimental  values 
of  x,  but  a  simpler  and  more  direct  approach  is  to 
regress  y  on  x.  That  is,  find  a  suitable  predic¬ 
tion  rule  or  function  which  can  be  used  to  relate 
the  independent  variable  of  x  relatively  inexpen¬ 
sive  units  of  flight  simulator  training  with  the 
dependent  variable  of  y  relatively  expensive  units 
of  aircraft  training  required  to  attain  the  train¬ 
ing  objective.  Once  this  training  effectiveness 
function  relating  units  of  simulator  training  with 
units  of  subsequently  required  aircraft  training 
is  determined,  the  training  psychologist  can  apply 
to  the  function  the  respective  cost  factors  asso¬ 
ciated  with  the  two  training  media  and  then 
minimize  the  resulting  total  cost  function. 

Derivation  of  Model 

Now  that  the  potential  utility  of  such  a 
concept  of  training  effectiveness  has  been  illus¬ 
trated,  how  can  the  function  relating  x  and  y  be 
characterized?  In  this  section  a  model  relating 
simulator  and  aircraft  training  will  be  developed, 
both  intuitively  and  theoretically,  and  evaluated 
against  extant  empirical  data. 

At  the  intuitive  level,  one  would  expect  the 
function  sought  to  exhibit  several  character¬ 
istics.  At  x  «  0,  i.e.,  no  simulator  training,  y 
is  equal  to  the  CTER's  yg,  the  amount  of  aircraft 
training  required  when  a  simulator  is  not  used. 

As  x  increases,  y  should  decrease;  that  is,  as 
amount  of  simulator  training  increases,  amount  of 
subsequent  required  aircraft  training  should 
decrease.  However,  the  rate  of  decrease  should 
not  be  constant;  from  the  nature  of  the  CTER,  it 
is  known  that  the  pay-back  from  investing  more 
units  of  training  in  the  simulator  becomes  less 
and  less.  That  is,  although  the  rate  of  change  of 
y  with  increasing  x  is  negative,  “a  rate  of 
change  approaches  zero,  resulting  i  some  asymp¬ 
totic  minimum  non-negative  value  of  y  which  will 
be  denoted  by  c.  The  value  of  c  represents  the 
amount  of  aircraft  training  that  must  be  done  to 
attain  the  training  objective  regardless  of  the 
amount  of  simulator  training  administered.  For 
the  training  effectiveness  model,  c  is  concep¬ 
tually  representative  of  those  "task  elements" 
which  are  not  trained  by  simulation  but  must  be 
learned  in  the  aircraft.  For  those  cases  in  which 
all  task  elements  are  trained  in  the  simulator,  c 
would  be  equal  to  zero.  An  intuitive  graph  of  tfie 
function  is  shown  at  Figure  1. 

Consider  y  as  it  ranges  between  a  maximum  at 
^  and  a  minimum  at  y  =  c.  The  quantity  yn  - 
c  can  be  considered  as  representative  of  the 
potential  aircraft  savings  that  can  be  realized  by 
using  the  simulator.  Assume  that,  as  x  increases, 
the  rate  at  which  y  decreases  (and  savings  accrue) 
is  a  constant  proportion  of  v  -  c.  This  can  be 
represented  mathematically  as  the  linear  differ¬ 
ential  equation 


d(y  -  c)  =  -b(y  -  c),  (2) 

dx 


where  b  is  the  proportional  constant.  Substi¬ 
tuting  £  for  y  -  c,  the  equation  becomes 

d(g)  =  -b(g) 
dx 


which  has  general  solution 


(3) 


where  a  is  an  arbitrary  constant.  Replacing  £  by 
y  -  c  yields 

y  -  c  =  ae‘bx  (4) 


y  =  ae‘bx  +  c. 


Equation  (4)  is  then  a  good  theoretical  candidate 
for  the  function  the  training  psychologist  seeks 
in  relating  simulator  training  with  aircraft 
training. 

Other  than  the  study  reported  here,  little 
quantitative  data  for  evaluation  of  the  model  are 
to  be  found  in  the  literature;  most  training 
effectiveness  studies,  being  oriented  toward 
transfer  of  training  proportions  or  toward  CTERs, 
have  not  systematically  varied  x,  the  amount  of 
simulator  training  given.  A  notable  exception 
occurs  in  a  study  conducted  by  Povenmire  and 
Roscoe  (3).  In  evaluating  a  generic  aircraft 
simulator,  Povenmire  and  Roscoe  gave  general 
aviation  students  up  to  11  hours  instruction  in 
the  simulator  followed  by  training  to  criterion  in 
the  aircraft.  Data  from  their  Table  3  are  plotted 
in  Figure  2.  The  curve  in  Figure  2  is  a  rough  fit 
of  equation  4  to  their  data.  For  this  fit,  the 
proportional  constant  b  has  an  approximate  magni¬ 
tude  of  .397,  which  is  well  within  its  theoreti¬ 
cally  expected  range. 


Figure  1.  Hypothetical  relation  between  ante 
cedent  simulator  training  and  subsequent 
required  aircraft  training. 
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Level  of  Analysis 

To  this  point,  no  mention  has  been  made  of  the 
specific  level  at  which  the  proposed  analysis  is 
to  be  made.  In  the  case  of  the  Povenmire  and 
Roscoe  data,  the  analysis  was  made  at  the  level  of 
the  entire  curriculum:  data  were  collected  in 
terms  of  the  total  time  students  were  trained  in 
the  simulator  or  in  the  aircraft.  Thus,  any 
measure  of  effectiveness  derived  is  a  measure  of 
the  training  device  as  a  whole.  However,  it  may 
be  that  the  device  is  more  effective  in  one  area 
of  training  than  in  another.  The  training  analyst 
requires  information  as  to  the  device's  areas  of 
greatest  effectiveness  in  order  that  training 
curricula  may  be  developed  which  capitalize  on  the 
simulator's  training  strengths.  One  way  this 
information  may  be  obtained  is  by  evaluating  the 
simulator  at  the  level  of  individual  training 
maneuvers.  In  using  this  level  of  approach  in  the 
evaluation  of  the  U.S.  Army's  CH-47  helicopter 
flight  simulator  (CH47FS),  Holman  (2)  found  that, 
although  the  CH47FS  is  effective  overall  (.82 
average  CTER),  CTERs  for  individual  maneuvers 
ranged  from  zero  to  2.80. 

In  view  of  this  finding,  and  since  the  AH1FS 
has  incorporated  in  it  most  of  the  CH47FS‘s  tech¬ 
nical  design  features,  it  was  decided  to  evaluate 
the  AH1FS  at  the  level  of  individual  maneuvers. 

The  general  approach  taken  was  to  administer  regu¬ 
lar  flight  students  varying  amounts  of  AH1FS 
training  in  each  maneuver  and  then  to  observe  the 
additional  amounts  of  aircraft  training  required 
for  them  to  attain  proficiency. 

PROCEDURE 

Subjects 

Instructors.  Instructor  pilots  (IPs)  then 
currently  assigned  to  the  Attack/Aeroscout  Branch 
of  Hanchey  Division  of  the  Department  of  Flight 
Training  of  the  Directorate  of  Training  of  the 
U.S.  Army  Aviation  Center  (USAAVNC)  served  as 
flight  instructors.  The  IPs  were  all  experienced 
aviators,  qualified  in  the  AH-1  aircraft,  and 
graduates  of  the  Attack/Aeroscout  Branch's  Methods 
of  Instruction  course, 


Figure  2.  Fit  of  model  to  Povenmire  &  Roscoe 
results. 


Students.  Experimental  subjects  (§s)  were 
rated  Army  rotary  wing  aviators  selected  from 
regular  USAAVNC  AH-1  transition  classes  in  resi¬ 
dence  during  April -August  1979.  Some  Ss  received 
all  their  training  in  the  AH-1  aircraft;  others 
received  AH1FS  training  followed  by  training  in 
the  aircraft. 

Administrative  constraints  and  scheduled  daily 
AH1FS  availability  restricted  the  experimental 
sample  size  per  class  to  8  for  aircraft  training 
only  and  to  6  for  simulator  plus  aircraft  training. 
Only  those  commissioned  officer  students  in  the 
grade  of  captain  or  below  and  those  warrant  officer 
students  in  the  grade  C43  or  below  were  considered. 
The  remaining  selection  criterion  was  number  of 
total  flight  hours:  those  Ss  with  the  lowest 
number  of  total  flight  hours  were  selected. 

Apparatus 

The  AH1FS  is  a  high-technology  training  device 
which  simulates  the  AH-1  aircraft  cockpit  and 
instrumentation,  aircraft  motion  and  vibration, 
aircraft  power  plant  and  weapons  noise,  and  out- 
the-window  view.  It  is  designed  to  afford  train¬ 
ing  in  visual  contact  flight,  instrument  flight, 
and  weapons  delivery  techniques. 

Preliminary  Activities 

Instructor  training.  Prior  to  start  of  the 
study  proper,  3  experienced  IPs,  selected  by  the 
Scout/Attack  Branch,  received  a  5-day  "instructor- 
operator"  course  conducted  by  the  simulator  manu¬ 
facturer.  Primary  topics  were  simulator  operating 
procedures  and  simulator-specific  instructional 
strategies.  At  the  end  of  training,  all  three 
were  judged  as  qualified  AH1FS  instructor-oper¬ 
ators  by  both  the  manufacturer  and  the  Army  simu¬ 
lator  test-acceptance  pilot  assigned  to  the 
project. 

Data  specifications.  As  indicated  above,  it 
was  decided  the  level  of  analysis  for  the  study 
would  be  that  of  the  individual  maneuver.  Prior 
to  the  study's  start,  the  suite  of  maneuvers  then 
taught  by  the  Scout/Attack  Branch  was  identified. 
Following  the  general  format  developed  by  Holman 
(2),  a  booklet  allowing  for  collecting  data  on  up 
to  four  repetitions  of  each  maneuver  daily  was 
developed.  Table  2  contains  a  list  of  the  maneu¬ 
vers  evaluated.  Data  collected  on  any  one  maneu¬ 
ver  repetition  included  the  number  of  training 
minutes  spent  in  performing  the  repetition  and  a 
rating  of  the  trainee's  overall  performance  on  the 
repetition.  The  scale  used  for  the  overall  rating 
is  shown  in  Figure  3. 

All  Scout/Attack  Branch  AH-1  IPs  were  instruct¬ 
ed  in  the  use  of  the  data  collection  booklet  and 
rating  scale.  Prior  to  commencement  of  the  test 
proper,  each  instructor  pilot  had  satisfactorily 
demonstrated  use  of  the  booklet  by  recording  data 
from  one  of  his  regular  transition  course  students. 

AHlFS  training  amount  determination.  As  1 ndi - 
cated  previously,  the  overall  test  methodology  was 
to  involve  observing  the  amount  of  aircraft  train¬ 
ing  required  after  various  amounts  of  AHlFS  train¬ 
ing.  Subsequent  regression  analysis  of  these  data 
would  require  that  the  amounts  of  AHlFS  training 
selected  be  Independent  of  the  trainees.  Specifi- 
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Bn 

Description 

0 

Demonstration  by  IP;  no  evaluation. 

1 

IP  immediately  had  to  take  back  control  of  the  aircraft. 

2 

Performance  deteriorated  until  IP  was  finally  obliged  to 
take  back  control  of  aircraft. 

3 

Student  required  considerable  verbal  assistance. 

4 

Some  parameters  within  course  limits;  verbal  correction 
from  IP  required. 

5 

Some  verbal  assistance  required;  less  than  half  of 
parameters  within  course  limits. 

6 

Minimal  verbal  assistance;  more  than  one-half  parameters 
within  course  limits. 

Few  parameters  outside  course  limits;  student  corrected 
performance  with  coaching;  still  lacks  good  control 
touch. 

8 

All  parameters  within  course  limits;  work  needed  on 
control  touch. 

9 

Outstanding;  no  perceptible  deviations  from  standards; 
SIP-level  performance. 

Figure  3.  Maneuver  rating  scale. 


cally,  in  such  a  regression  analysis,  Ss  cannot 
have  been  trained  to  some  level  of  proficiency; 
having  done  so,  in  effect,  would  have  allowed  each 
trainee  to  determine  his  own  amount  of  AH1FS 
training  and  thereby  violate  the  underlying  assump¬ 
tion  of  independent  assignment  of  training  amounts. 
Thus  it  was  determined  that  for  each  maneuver,  Ss 
would  each  receive  one  of  3  pre-specified  numbers 
of  training  repetitions  in  the  AHIFS. 

Inspection  of  Figure  1  indicates  that  the 
magnitudes  of  the  3  values  chosen  can  be  critical 
to  the  analysis.  If  all  3  independent  variable 
values  are  chosen  too  large,  then  the  resulting 
dependent  variable  values  will  all  lie  in  the 
asymptotic  portion  of  the  curve  and  inferences 
about  the  descending  portion  of  the  curve  may  lack 
precision.  On  the  other  hand,  if  all  3  values  are 
chosen  too  small,  then  the  dependent  variable 
values  will  all  lie  in  the  descending  portion  of 
the  curve  and  inferences  about  the  magnitude  of 
the  asymptote  may  lack  precision.  It  can  be  seen 
that,  ideally,  independent  variable  values  for 
each  maneuver  should  be  chosen  such  that  resultant 
dependent  variable  values  fall  both  in  the  descen¬ 
ding  and  in  the  asymptotic  portions  of  the  curve. 

Since  the  AH1FS  was  a  new  piece  of  equipment 
with  no  quantitative  training  effectiveness  his¬ 
tory,  estimation  of  each  maneuver's  ideal  amounts 
of  training  was,  of  necessity,  based  on  several 
outside  considerations.  First,  Scout/  Attack 
Branch  IPs  were  asked  to  estimate  the  average 
number  of  maneuver  repetitions  the  average  AH-1 
transition  course  student  requires  to  reach 
institutional  proficiency  in  the  aircraft.  Also, 
for  maneuvers  common  to  both  the  AH-1  and  CH-47, 
data  collected  in  the  CH-47  flight  simulator  eval¬ 


uation  (2)  were  examined.  Then  based  on  these 
data,  on  their  sizable  experience  as  IPs,  and  on 
their  perceived  effectiveness  of  the  AH1FS  as  a 
training  device,  the  AH1FS  instructor  pilots  and 
the  simulator  project  test  pilot  individually  and 
then  collectively  estimated  for  each  maneuver 
three  amounts  of  AH1FS  training  that  should  cap¬ 
ture  both  the  descending  and  the  asymptotic  por¬ 
tions  of  the  generic  curve  shown  in  Figure  1. 

Method 

Due  to  various  operational  considerations,  it 
was  decided  that  Ss  trained  in  the  AH1FS  would 
receive  at  least  some  training  on  all  maneuvers  in 
the  simulator;  data  for  each  maneuver  for  the 
condition  "no  AH1FS  training"  were  to  be  collected 
from  Ss  receiving  all  their  training  in  the  AH-1 
aircraft.  The  normal  AH-1  transition  course  as 
taught  at  USAAVNC  had  a  maximum  If  12  students  and 
was  of  4  week's  duration,  with  a  new  class  start¬ 
ing  every  2  weeks.  Ss  to  receive  AH1FS  training 
were  selected  from  every  other  class;  Ss  to 
receive  aircraft  training  only  were  seTected  from 
each  class  as  feasible. 

AHIFS  training.  Simula  tor-trained  Ss  followed 
the  same  general  daily  training  routine  as  their 
aircraft-trained  counterparts.  The  standard  daily 
routine  allowed  for  two  instructors  each  to  train 
3  students  for  1.5  hours  apiece.  Except  for 
utilization  of  simulator-specific  features  such  as 
"freeze,”  "play-back,"  and  demonstration  tapes, 
the  AHIFS  Instructor  pilots  followed  the  same 
standard  curriculum  that  was  being  used  in  the 
aircraft.  The  only  major  departure  from  the 
standard  was  in  progression  through  the  curriculum: 
where  individual  training  progression  was  profi- 
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TABLE  1.  STUDENT  GENERAL  CHARACTERISTICS 


Low 

Averaqe 

High 

1. 

Age 

21 

27 

32 

2. 

Total  RW  flight  hours 

160 

594 

2500 

3. 

Total  RH  flight  hours 

0 

95 

190 

in  last  6  months 

4. 

Years  since  graduation 

0 

2.3 

9 

from  RW  flight  school 

ciency-based  in  the  aircraft,  in  the  AH1FS  it  was 
based  on  completion  of  the  pre-speci fied  numbers 
of  training  iterations  of  each  maneuver.  For  each 
S  for  each  maneuver,  the  pre-speci fied  level  of 
training  to  be  received  was  assigned  randomly 
under  the  constraint  that  overall  equal  numbers  of 
Ss  received  each  of  the  3  levels.  After  comple¬ 
tion  of  AH1FS  training,  Ss  began  training  in  the 
aircraft. 

Aircraft  training.  The  AH 1FS- trained  S's 
first  exposure  to  the  AH-1  aircraft  was  a  diagnos¬ 
tic  checkride  administered  by  a  Standardization 
Instructor  Pilot  from  the  USAAVNC  Directorate  of 
Evaluation/Standardization.  Based  on  the  results 
of  this  checkride,  the  S's  AH1FS  instructor  con¬ 
tinued  training  him  to  proficiency  in  the  AH-1 
aircraft.  When  his  instructor  considered  him 
proficient  in  the  aircraft,  the  student  was  given 
an  end  of  course  aircraft  checkride  by  another  IP 
and  released  from  training. 

Those  Ss  not  receiving  training  in  the  AH1FS 
received  normal  instruction  and  training  in  the 
aircraft. 

RESULTS 

Subjects 

During  the  conduct  of  the  study,  the  Scout/ 
Attack  Branch  experienced  unforseen  shortages  of 
both  personnel  and  aircraft.  The  effects  upon  the 
study  were  two-fold:  new  instructor  pilots  with 
no  experience  with  the  data  collection  booklet 
entered  the  training  system,  and  students  trained 
in  the  aircraft  many  times  received  aircraft 
instruction  from  more  than  one  instructor  pilot. 

As  new  Instructor  pilots  began  carrying  students, 
they  were  Instructed  in  the  use  of  the  data  col¬ 
lection  booklet  and  began  collecting  data  on  their 
students.  A  new  Instructor  pilot’s  first  students' 
data  were  discarded.  Also,  any  student  receiving 
Instruction  from  more  than  two  instructor  pilots 
(not  counting  the  checkride  IPs)  during  the  four- 
week  transition  course  was  discarded  from  the 
analysis. 

Students ■  A  total  of  22  Ss  began  training  in 
the  AH1FS.  Ml th  the  exception  of  one  who  was 
grounded  for  medical  reasons  unrelated  to  the 
test,  all  successfully  completed  AH-1  flight 
training.  A  total  of  25  Ss  entered  the  study  to 
receive  aircraft  training  only.  Due  to  the  above- 
mentioned  problems  with  Instructor  availability, 
data  from  all  but  14  of  these  Ss  were  discarded. 


Descriptive  data  of  these  35  Ss  is  shown  in  Table 

1. 

Maneuvers 

Missing  data.  If  in  simulator  training  a  £ 
received  as  many  as  2  fewer  or  as  many  as  2  more 
training  repetitions  for  a  maneuver  than  had  been 
assigned  him,  his  data  for  that  maneuver  were 
discarded.  This  condition  generally  arose  due  to 
abnormally  low  simulator  availability  or  through 
oversight  on  the  part  of  the  simulator  IPs.  Also, 
for  some  maneuvers,  some  Ss  trained  in  the  air¬ 
craft  alone  were  neither  trained  to  criterion  (as 
defined  below)  nor  tested  on  that  maneuver  on  the 
end-of-course  checkride.  Data  in  these  cases  were 
also  discarded.  Thus,  in  most  of  the  results 
given  below,  data  for  a  maneuver  are  based  on  a 
sample  of  less  than  35. 

Over-training.  It  was  discovered  early  in  the 
Study  that,  although  the  AH-1  transition  course 
was  (within  the  limits  of  its  4-week  duration) 
self-paced  and  proficiency  based,  over-training 
unavoidably  occurred  on  some  maneuvers.  For 
example,  since  most  training  involving  takeoffs 
and  landings  or  autorotations  Involved  flying  a 
traffic  pattern  around  the  training  stagefield, 
students  in  the  aircraft  routinely  received  con¬ 
siderable  over-training  in  flying  traffic  pat¬ 
terns.  Thus,  after  consulting  with  all  the  in¬ 
structors  involved,  it  was  decided  that  a  student 
would,  for  purposes  of  the  study,  be  considered  to 
have  attained  proficiency  on  a  maneuver  in  the 
aircraft  when  for  3  consecutive  training  repeti¬ 
tions  he  had  been  rated  at  least  a  "7"  (see  Fig. 

3)  and,  of  course,  provided  he  was  rated  at  least 
a  "7"  on  the  maneuver  on  the  end-of-course  check¬ 
ride.  All  aircraft  training  subsequent  to  the 
three  "7s1'  criterion  was  considered  over-training 
and  not  included  in  the  analysis  below. 

Presence  of  trend.  As  a  general  indicant  of 
degree  of  overall  relationship  between  amount  of 
AH1FS  training  and  subsequent  required  AH-1  air¬ 
craft  training,  eta-squared  was  computed  for  the 
data  for  each  maneuver.  The  values  found,  which 
can  for  this  sample  of  Ss  be  Interpreted  as  the 
proportion  of  variance  accounted  for  by  knowledge 
of  amount  of  AH1FS  training,  are  entered  in  Table 
2. 

Regression  analysis.  For  each  maneuver,  the 
data  described  above  were  fit  to  the  function  f(x) 
=  ae"bx  +  c  using  the  SPSS  sub-program  NON-LINEAR 
(4).  Marquardt's  method  was  used  to  obtain 
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TABLE  2.  MANEUVER  RESULTS 


Parameter  Estimates 


Maneuver 

N 

n2 

a 

b 

c 

df 

F-ratio 

Residual 

Cockpit  procedures 

34 

.82 

6.41 

1.878 

2.65 

1,30 

3.711 

1.39 

Takeoff  to  hover 

32 

.56 

10.24 

3979 

3.68 

1,28 

.074 

4.00 

Hover  flight 

33 

.59 

11.87 

.368 

3.35 

1,29 

.033 

4.87 

Hover  landing 

35 

.71 

10.23 

.390 

2.98 

1,31 

.029 

3.14 

Hi -speed  flight 

33 

.60 

6.89 

1.129 

3.03 

1,29 

.317 

2.71 

Normal  takeoff 

33 

.35 

13.86 

.161 

2.57 

1,29 

.198 

4.53 

Normal  approach 

33 

.79 

9.96 

.131 

2.02 

1,29 

.696 

2.06 

Maximum  power  takeoff 

29 

.63 

4.56 

.400 

1.16 

1,25 

.061 

1.59 

Steep  approach 

30 

.44 

2.88 

.724 

1.58 

1,26 

1.243 

1.60 

Running  landing 

23 

.64 

1.95 

.436 

.78 

1,19 

1.180 

.64 

Traffic  pattern 

33 

.64 

16.36 

.608 

4.57 

1,29 

.024 

6.05 

Hydraulics  failure 

Forced  landing. 

27 

.72 

8.18 

.179 

1.54 

1,23 

.036 

2.84 

power  recovery 
Autorotation  to 

27 

.53 

8.35 

1.5xl06 

3.15 

1,23 

.038 

3.92 

touchdown 

31 

.53 

8.34 

5 . Ixl03s 

4.44 

1,27 

.099 

3.85 

Autorotation  with  turn 
Autorotation,  power 

33 

.29 

4.57 

.395 

3.37 

1,29 

.477 

3.47 

termination 

19 

.05 

.50 

•17.20 

2.00 

1,15 

.>>09 

1.08 

Hovering  autorotation 
Left  anti -torque 

32 

.49 

3.59 

1.7xl054 

2.06 

1,28 

1.991 

1.95 

failure 

Right  anti -torque 

28 

.61 

8.52 

.316 

1.55 

1,24 

.418 

3.38 

failure 

27 

.53 

6.99 

.548 

1.51 

1,23 

1.561 

2.90 

Low  level  autorotation 
Low  level  high  speed 

28 

.43 

3.57 

2.5x10s 

3.20 

1,24 

.928 

2.16 

autorotation 

Hover  out  of  ground 

26 

.24 

1.62 

1743 

2.54 

1,22 

1.330 

1.60 

effect 

25 

.56 

1.90 

2.2x10s 

1.25 

1,21 

3.654 

1.04 

Terrain  flight  takeoff 

23 

.48 

3.73 

1.388 

.27 

1,19 

.046 

1.67 

Terrain  flight 

Terrain  flight 

26 

.44 

3.81 

1.149 

.55 

1,22 

.214 

1.96 

approach 

22 

.36 

3.29 

17.16 

1.00 

2,18 

.229 

2.39 

SCAS  off  operations 
Weapons  cockpit 

22 

.09 

.10 

1.555 

2.16 

1,18 

1.980 

1.06 

procedures 

2.75"  FFAR  ballistic 

25 

.49 

4.54 

478.7 

.38 

1.21 

.651 

2.61 

correction 

27 

.46 

5.16 

5.2x10' 3 

.07 

1,23 

.020 

2.77 

FFAR  firing 

20nm  ballistic 

28 

.49 

5.32 

4.2xl043 

.07 

1,24 

.012 

2.69 

correction 

27 

.74 

2.16 

3180 

.07 

1,23 

.147 

.65 

20mm  firing 

24 

.65 

2.38 

1.6xl07 

.08 

1,20 

.102 

.86 

parameter  estimates;  iteration  ceased  when  the 
largest  relative  change  among  the  3  parameters 
became  less  than  1.5xlCT8. 

Best  fit  parameters  for  each  maneuver  are 
entered  in  Table  2.  Also  shown  for  each  Is  its 
RMS  residual  and  an  F-ratlo  of  goodness-of-fit 
(5).  In  the  interest  of  conserving  space,  graphic 
results  for  only  4  maneuvers  are  shown;  these 
appear  in  Figures  4-7  and  are  discussed  in  more 
detail  below. 

DISCUSSION 

As  indicated  at  the  outset,  one  of  the  major 
objectives  of  the  test  of  the  AH1FS  was  to  evalu¬ 
ate  a  methodology  for  quantifying  training  and 
cost  effectiveness  of  simulators.  In  this  sec¬ 
tion,  the  success  of  the  methodology  In  capturing 
the  simulator's  effectiveness  will  be  scrutinized. 


and  then  a  straightforward  application  of  the 
results  to  curriculum  development  will  be  outlined. 

Training  Effectiveness 

The  overall  data  indicate  that  the  AH1FS  is 
an  effective  device  in  training  nearly  all  maneu¬ 
vers  investigated.  This  is  evidenced  by  the 
general  reduction  in  required  aircraft  training 
for  each  maneuver  following  simulator  training 
(l.e.,  for  all  maneuvers,  the  rate  parameter  b 
indicates  a  decreasing  function  of  ^).  But  the 
specific  results  of  particular  Interest  are  those 
pertaining  to  the  accuracy  of  the  transfer  model 
and  the  success  of  the  methodology  in  obtaining 
usable  input  for  efficient  curriculum  design. 

Presence  of  trends.  The  initial  analysis  of 
the  data  indicates  there  is  Indeed,  in  most  cases, 
a  functional  relationship  to  be  found  between 
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Aircraft  trials 


5  10  15  20  25 


Simulator  trials 
Figure  4.  Hydraulics  failure. 


1  2  3  4  5 


Simulator  trials 
Figure  6.  20mm  firing. 


amount  of  simulator  training  and  amount  of  subse¬ 
quent  aircraft  training.  Table  2  shows  that  for 
this  sample,  except  for  "autorotation,  termination 
with  power"  and  "SCAS  off  operations,"  between  25 
and  80  percent  of  the  variance  in  aircraft  train¬ 
ing  amounts  can  be  accounted  for  by  amount  of 
simulator  training,  depending  on  the  maneuver. 

Hence,  there  is  some  motivation  for  attempting  to 
fit  a  model  to  the  data. 

Goodness-of-flt.  In  all  cases,  the  model  fits 
the  data  for  each  maneuver  with  values  for  a  and  c^ 
within  their  theoretically  expected  ranges. 

(Problems  with  values  for  b  will  be  discussed 
below.)  However,  It  is  difficult  to  judge  the 
absolute  goodness  of  fit  of  any  model.  For  the 
curve-fitting  routine  used  with  most,  parameter 
values  are  selected  that  maximize  the  precision 
with  which  the  dependent  variable  can  be  predicted 

216 


1  2  3  4  5  6 


Simulator  trials 

Figure  5.  Autorotation,  termination  with 
power. 


5  10  15  20  25 


Simulator  trials 
Figure  7.  Takeoff  to  a  hover. 


from  the  independent  variable.  Of  course,  the 
predicted  values  and  their  corresponding  observed 
dependent  variable  values  will  differ;  the  magni¬ 
tude  of  the  variance  of  this  difference  is  a 
general  indicant  of  goodness  of  fit:  small  vari¬ 
ance  results  from  a  good  fit.  However,  it  is  also 
the  case  that  for  a  given  level  of  the  independent 
variable,  there  will  be  variance  in  the  resultant 
levels  of  dependent  variable  observed.  If  this 
variance  is  conceptualized  as  the  "noise"  inherent 
in  the  data,  then  at  least  that  much  "noise"  is 
also  to  be  expected  in  the  precision  of  prediction 
using  the  best  fit  parameters.  To  the  extent  the 
variance  of  the  fit's  precision  exceeds  that  of 
the  dependent  variable  observed  values,  the  fit 
can  be  regarded  as  bad.  Conversely,  a  good  fit 
will  yield  a  precision  variance  not  significantly 
larger  than  the  dependent  variable  variance.  The 
F-ratios  in  Table  2  compare  these  variances;  none 


i 


is  significant  at  the  a  =  .05  level.  But  this  is 
somewhat  to  be  expected:  the  experimental  design 
could  economically  allow  sampling  at  only  4  values 
of  the  independent  value,  and  the  model  is  left  to 
fit  the  4  resulting  mean  dependent  variable  values 
with  3  free  parameters.  It  should  be  pointed  out 
that  a  "good"  fit  is  not  necessarily  "the"  fit; 
there  are  other  models  and  theoretical  functions 
that  would  fit  the  data  just  as  well  or  even 
better.  For  example,  Cronholm  (6)  has  pointed  out 
that,  if  both  simulator  and  aircraft  learning 
curves  are  assumed  exponential  with  rate  parame¬ 
ters  a  and  h,  respectively,  then  a  very  good  case 
can  be  made  for  a  function  of  the  form 


y  =  (h*1)  In  (ae*gx  +  c). 


Thus  it  may  only  be  concluded  that  there  is  no 
cogent  reason  for  rejecting  as  a  viable  heuristic 
the  model  under  consideration. 

Individual  maneuvers.  The  maneuver  "best 
fit"  curves  "obtained  can  be  arbitrarily  placed  in 
four  categories  represented  by  Figures  4-7. 

The  first  category,  of  which  Figure  4,  "hy¬ 
draulics  failure,"  is  an  example,  consists  of 
those  "well-behaved"  curves  for  which  a  is  large 
relative  to  c  and  the  rate  parameter  b  is  not  too 
precipitous.  As  inspection  of  Table  2  parameter 
values  shows,  this  was  the  case  with  the  majority 
of  the  maneuvers  evaluated. 

A  second  category,  represented  by  Figure  5, 
"autorotation,  termination  with  power,"  consists 
of  those  maneuvers  for  which  a^  is  small  relative 
to  c.  This  relationship  is  indicative  of  poor 
transfer  to  the  aircraft  and  occurred  with  only 
one  other  task,  "SCAS-off  operations." 

A  third  category,  represented  by  Figure  6, 
"20mm  firing,"  is  characterized  by  values  of  c 
close  to  zero.  A  negligible  value  of  c  can,  as 
previously  discussed,  be  considered  indicative 
that  complete  transfer  from  simulator  to  aircraft 
was  attained.  The  five  gunnery  tasks  and  "terrain 
flight  take-off"  fall  into  this  category. 

The  last  category  is  represented  by  Figure  7, 
"take-off  to  a  hover."  As  mentioned  previously. 
Ideally,  experimental  values  of  the  independent 
variable  should  be  chosen  such  that  not  all  three 
yield  dependent  variable  values  falling  in  the 
asymptotic  portion  of  the  curve.  Despite  the 
attempts  made  to  avoid  them,  such  choices  were 
evidently  made  in  the  cases  of  "take-off  to  a 
hover,"  "forced  landing  with  power  recovery," 
"touch-down  autorotation,"  "hovering  autorota¬ 
tion,"  "low  level  autorotation,"  "low  level  high 
speed  autorotation,"  and  "hover  out  of  ground 
effect." 

Consider  Figure  7.  Based  on  various  consid¬ 
erations,  it  was  decided  to  sample  the  independent 
variable  at  levels  of  0,  13,  19,  and  24  repeti¬ 
tions.  Inspection  of  Figure  7's  best  fit  curve 
indicates  that  13  or  more  AH1FS  training  repeti¬ 
tions  yield  results  in  the  asymptotic  area  of  the 
function.  The  reiterative  curve-fitting  routine 
used,  in  effect,  fit  these  asymptotic  data  with  a 
line  parallel  to  the  abscissa.*  Although  it  may 


be  that  one  training  repetition  in  the  AH1FS  is 
efficacious  to  the  extent  indicated  by  the  curve, 
it  is  very  much  more  likely  that,  had  an  indepen¬ 
dent  variable  value  in  the  range  of  3-7  repeti¬ 
tions  been  chosen,  a  much  less  acute  function 
would  have  been  obtained. 

Integration  of  Cost  with  Training  Effectiveness 

Since  costing  procedures  for  both  the  simula¬ 
tor  and  the  aircraft  are  carried  out  in  terms  of 
hours  of  operation  rather  than  in  numbers  of 
maneuver  repetitions,  prior  to  integrating  cost 
and  training  effectiveness,  the  training  analyst 
should  determine  the  average  time  required  per 
maneuver  in  each  training  device.  Then  the  appro¬ 
priate  transformations  of  axes  of  curves  such  as 
those  in  Figures  4  through  7  can  be  made  such  that 
the  abscissae's  unit  of  measure  is  units  of  simu¬ 
lator  time  and  the  ordinates'  unit  of  measure  is 
units  of  aircraft  time. 

Device  operation  costs.  To  determine  economi¬ 
cal  !>TopITmalmTxis_o?^iTmulator  and  aircraft 
time,  the  training  analyst  must  be -provided  the 
cost  per  unit  of  operating  time  for  both  devices. 
If  this  figure  for  the  simulator  is  designated  Cj, 
then  the  function  describing  the  total  cost  of  x 
units  of  simulator  training  time  will  be  the 
product  CjX,  as  shown  in  Figure  8a.  Likewise,  if 
after  x  units  of  simulator  training,  y  units  of 
aircraft  training  are  required,  the  total  cost  of 
this  training  will  be  the  product  C^y,  where  is 
the  cost  per  unit  of  aircraft  training  time. 

Since  it  has  been  shown  that  the  y  units  of  re¬ 
quired  aircraft  training  time  can  be  expressed  as 
a  function  of  x  as  ae~bx  +  c,  then  the  cost  of 
required  aircraft  training  can  be  more  explicitly 
expressed  as  CA(ae‘bx  +  c)  as  in  Figure  8b.  Then, 
for  any  one  maneuver,  the  total  cost  can  be  ex¬ 
pressed  as 

C  =  Csx  +  CA(ae'bx  +  c),  (5) 

which  is  illustrated  graphically  in  Figure  8c. 

Inspection  of  Figure  8c  indicates  it  has  a 
point  at  which  total  cost  is  minimized,  and  it  can 
be  shown  mathematically**  that  equation  5  is  at  a 
minimum  when 

x  =  (lnCA  +  In  a  +  In  B  -  In  C5)  (b”M  (6) 

If  for  any  maneuver  m  this  optimal  value  of  x.  is 
denoted  as  x^,,  then  total  optimized  training  cost 
for  all  M  maneuvers  can  be  expressed  as 


M 


m=l 


Cc>C 
S  m 


e'bm*iii  + 


Cm> 


(7) 


*  Note  that  for  moderately  large  values  of  b,  the 
value  of  ae*DX  very  quickly  approaches  zero. 

**  dC  =  Cs  -  CAabe‘bx,  which,  when  set  at  zero  and 
3x  solved  for  x,  yields  equation  6. 


d^C  >  0  (given  a  >  0  and  b  >  0)  Implies  x  in 
4*1  equation  6  represents  a  minimum. 
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and  the  raw  savings  realized  will  be,  of  course, 
the  difference  between  Cj  evaluated  at  x  =  0  and 
the  Cj  of  equation  7. 

Thus  it  can  be  seen  that  the  model  and  method¬ 
ology  presented  here  are  both  viable  and  of  great 
utility  to  the  training  analyst. 

Further  Considerations 

Although  the  methodology  presented  here  is 
fairly  straightforward,  there  are  some  additional 
factors  to  be  kept  in  mind  in  the  details  of 
applying  its  results  in  development  of  a  simula¬ 
tor-based  training  system. 

Effects  of  curriculum.  It  is  an  inescapable 
fact  that,  regardless  of  the  level  of  technology 
and  the  sophistication  of  a  simulator,  its  effec¬ 
tiveness  is  a  function  of  how  it  is  used,  of  how 
the  trainer  incorporates  its  features  into  a 
training  program.  The  quantitative  measures  of 
effectiveness  determined  by  this  study  are  very 
much  a  function  of  how  the  IPs  used  the  simulator 
as  a  training  device.  As  the  simulator  instruc¬ 
tional  tactics  are  refined,  the  AHIFS's  effective¬ 
ness  should  improve.  Considered  in  this  light, 
the  trade-off  curves  determined  by  the  study 
represent  not  the  optimum  effectiveness  of  the 
device,  but  the  baseline  effectiveness. 

Restricted  device  availability.  As  mentioned 
in  the  introduction,  most  modern  simulators  are 
high  cost  assets  and  must  be  distributed  over  a 
large  number  of  trainees.  In  all  cases,  the 
training  analyst  will  have  the  device  available 
for  some  finite  period  each  day.  In  many  cases 
the  analyst  is  also  faced  with  cycling  through  the 
curriculum  a  large  number  of  students  within  a 
fixed  number  of  training  days.  These  restrictions 
determine  the  amount  of  simulator  time  that  will 
be  available  to  each  student.  Ideally,  the  amount 
of  time  allocable  per  student  would  be  at  least 
equal  to  the  total  optimal  maneuver  training 
times.  In  many  Instances,  this  Is  not  the  case 
and  less  than  optimal  training  curricula  must  be 
set  up  according  to  some  trade-off  scheme. 

In  his  implementation  of  the  integration  pro¬ 
cedures  outlined  here,  Hopkins  (7)  ordered  the 
simulator  trainable  maneuvers  in  terms  of  each 
maneuver's  savings  per  hour  of  simulator  opera¬ 
tion.  (In  general,  more  savings  per  unit  of 
simulator  operating  time  accrue  to  those  maneuvers 
for  which  the  difference  a  -  c  is  great  and  the 
rate  parameter  b  is  largeT)  Choosing  a  hypothe¬ 
tical  3.5  hour  availability  per  student,  he  simply 
cumulated  the  x'  values  (in  terms  of  time)  down 
the  rank-ordered  list  of  maneuvers  until  they 
totalled  3.5  hours.  Other  trade-off  schemes  might 
alternatively  involve  such  considerations  as 
weighting  each  maneuver  according  to  the  danger 
associated  with  performing  it  in  the  aircraft. 

Conclusions 

At  the  practical  level,  the  model  and  method¬ 
ology  have  been  demonstrated  as  both  viable  and  of 
utility.  The  model  does  not  concern  Itself 
directly  with  such  issues  as  fidelity  and  realism, 
but  addresses  directly  the  effectiveness  of  the 
simulator  in  decreasing  required  aircraft  training 
time.  Aircraft  training  time  is  expressed  as  a 
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a.  Cumulative  FS  training  cost  as  a  function  of 
FS  training  time. 


FS  training  time 

b.  Cumulative  aircraft  training  cost  as  a 
function  of  FS  training  time. 


c.  Cumulative  total  cost  as  a  function  of  FS 
training  time. 

Figure  8.  Integration  of  simulator  and  aircraft 
training  costs. 
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sum  of  two  factors;  the  model  can  be  easily  ex¬ 
panded  to  Include  other  factors  such  as  student 
experience  or  aptitude  or  cumulative  negative 
effects  of  simulator  training.  For  mathematical 
rigor,  the  number  of  levels  of  the  Independent 
variable  sampled  should  be  at  least  one  more  than 
the  number  of  free  parameters  estimated.  With 
careful  selection  of  levels  of  the  Independent 
variable,  there  is  no  requirement  that  some  Ss  be 
used  for  "control"  data  and  receive  no  simulator 
training. 
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ABSTRACT 

Training  effectiveness  of  Navy  Device  2B3C,  a  computer  Image  visual  system  used  In  Navy 
Advanced  Jet  Phase  UPT,  was  evaluated  utilizing  a  transfer  of  training  design.  Comparisons 
were  made  for  two  simulator  groups  and  a  control  group  for  the  Familiarization,  Night 
Familiarization,  and  Carrier  Qualification  stages  of  training;  for  the  weapons  stage 
training,  comparisons  were  made  for  a  single  simulator  group  and  a  control  group.  Results 
are  presented  separately  for  the  various  training  stages.  Implications  for  Navy  UPT,  the 
Navy  VTXTS  procurement,  Instructor  training,  and  for  visual  simulation  In  general  are  drawn. 


I.  INTRODUCTION 

The  U.S.  Navy  was  the  first  of  the  military 
services  to  exploit  visual  flight  simulation  for 
Undergraduate  Pilot  Training  (UPT).  In  1972,  a 
prototype  wide-angle  computer-image  visual  display 
system  was  Installed  on  a  2F90  operational  flight 
trainer  (OFT)  for  the  TA-4J  Advanced  Jet  phase 
training  aircraft.  Following  an  extensive 
evaluation  (1)  of  Its  potential  for  enhancing  that 
phase  of  Navy  UPT,  decision  was  made  to  procure  two 
additional  devices.  This  allowed  one  such  visual 
trainer  at  each  of  the  Navy's  three  Advanced  Jet 
training  sites.  The  two  production  devices, 
slightly  modified  versions  of  the  original 
prototype,  were  designated  Device  2B35  and  were  put 
Into  operational  use  In  1976. 

Coincident  with  their  Installation,  the 
Advanced  Jet  syllabus  was  revised  (2)  to  provide 
approximately  six  hours  In  the  visual  trainer 
during  the  Famll larlzatlon  Stage  (FAM),  eight  hours 
of  device  training  at  the  start  of  the  Weapons 
Stage  (WEP),  and  a  brief  session  for  practice  of 
carrier  deck  emergency  procedures  prior  to  Carrier 
Qualification  (CQ).  It  is  Interesting  to  note  that 
any  students  n*!0  do  not  receive  the  prescribed  FAM 
stage  device  flights,  due  to  device  down-time  or 
other  factors,  are  given  two  extra  aircraft  rides. 
However,  students  missing  the  WEP  stage  device 
training  are  not  provided  any  added  air  time. 

Device  2B35  and  Its  antecedent  prototype  are 
Computer  Image  Generation  (CIG)  visual  systems 
representative  of  early  CIG  visual  technology.  The 
visual  system  presents  a  60°  X  210"  field  of  view 
by  means  of  three  rear-projection  screens,  each  of 
which  Is  Illuminated  by  a  light-valve  projector. 
The  scene  Is  full  color.  The  Instructor-operator 
console  Is  relatively  unsophisticated,  having  the 
visual  trainer  controls  and  repeater  Instruments,  a 
three-monitor  TV  display  of  the  visual  scene,  and  a 
teletype  printout  of  selected  performance 
parameters,  such  as  bomb  distance  data,  carrier 
wire  engagement,  and  the  like. 


The  2B35  visual  system  is  integrated  with  a 
Device  2F90  cockpit.  The  2F90  OFT  was 
Introduced  in  the  late  1960s.  It  has  a  limited 
pitch,  roll,  and  heave  motion  system  and  is  used  in 
teaching  instrument  flight  procedures.  Because  the 
CIG  visual  system  and  the  2F90  device  to  trfiich  It 
has  been  attached  are  inseparable  *#ien  used  in 
training,  further  references  in  this  paper  to  the 
2B35  will  Include  both  the  visual  system  and  the 
host  2F90  OFT. 

As  they  gained  operational  experience  with  the 
2B35,  the  Navy  personnel  became  increasingly 
concerned  about  the  real  training  worth  of  the 
device.  While  the  training  personnel  were 
generally  convinced  that  there  was  positive 
training  value  from  the  2B35  for  WEP  training,  they 
were  equally  convinced  that  there  were  serious 
undesirable  effects  from  device  use  during  FAM 
training.  The  possibility  of  negative  transfer  In 
FAM  was  of  particular  concern,  because  the  three 
sessions  In  the  device  were  being  used  to  replace 
two  aircraft  FAM  flights. 

Concurrently,  there  was  growing  Interest 
DoD-wlde  In  formal  evaluation  of  the  training 
effectiveness  of  all  aircrew  training  devices,  and 
of  flight  simulators  In  particular.  This  position 
was  endorsed  by  the  Chief  of  Naval  Operations 
( CNO) .  It  was  in  response  to  CNO  direction,  and  in 
recognition  of  the  user  concerns  over  Device  2B35's 
training  effectiveness,  that  the  Chief  of  Naval 
Education  and  Training  (CNET)  decided  to  undertake 
a  rigorous  appraisal  of  the  training  worth  of  the 
2B35  by  means  of  a  transfer  of  training  study.  The 
Navy  subsequently  contracted  with  Seville  Research 
Corporation  to  conduct  such  a  study. 

The  Transfer  of  Training  Effectiveness 
Evaluation  (TZEZ)  study  was  to  be  conducted 
as  a  joint  Navy-contractor  effort.  Seville 
would  be  responsible  for  the  study  design, 
preparation  of  Implementing  instructions  and 
material,  data  analyses,  and  reporting  of  results. 
The  Navy  retained  responsibility  for  the  actual 
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administration  of  all  Instruction  and  data 
collection,  although  Seville  was  to  maintain  close 
surveillance  over  these  Navy  activities.  Navy 
guidelines  for  the  conduct  of  the  study  were  that 
Seville  develop  an  evaluation  design  trfilch  would 
assure  a  fair  appraisal  of  the  2B35's  training 
worth,  but  It  was  also  required  that  the  evaluation 
would  be  minimally  disruptive  of  ongoing  training 
activities. 

This  paper  reports  the  results  of  the 
consequent  T*e’  study.  It  summarizes  the  necessary 
preparatory  activities,  briefly  describes  the  study 
design,  reports  the  principal  effects  of  device 
training,  and  evaluates  the  probable  utility  of  the 
2B35  within  the  Navy's  Advanced  Jet  pipeline.  It 
also  makes  certain  observations  regarding  the  use 
of  visual  flight  simulation  In  future  Navy  Jet 
Undergraduate  Pilot  Training. 


II.  METHOD 

The  study  began  with  a  review  of  the  content 
and  management  of  Navy  jet  training  and  an 
assessment  of  the  2B35's  potential  for  training. 
These  activities  were  accomplished  through  review 
of  Chief  of  Naval  Air  Training  (CNATRA)  syllabi  and 
training  materials,  Interactions  with  the  CNATRA 
staff,  and  on-site  observations  of  Advanced  Jet 
training  operations  at  Training  Wing  3,  NAS  Chase 
Field,  Texas,  the  site  selected  by  the  Navy  for  the 
study . 

Device  2B35  Training  Potential 

It  quickly  became  apparent  that  there  were 
fundamental  problems  with  the  2835  that  would  have 
to  be  solved  before  the  transfer  study  could  begin. 
Observation  of  students  and  instructors  flying  the 
2B35  convinced  the  study  team  that  It  was  neither 
providing  proper  cues  nor  eliciting  the  correct 
responses.  The  two  most  serious  problems,  close-in 
ball  and  power  control  and  carrier-deck  trapping, 
were  believed  due  to  errors  In  basic  modeling  which 
made  successful  performance  of  these  tasks  nearly 
Impossible.  User  suspicions  of  negative  transfer 
on  tasks  Involving  these  device  characteristics 
appeared  justified.  Fortunately,  a  priority 
corrective  action  program  Initiated  by  CNATRA  and 
engineers/programmers  from  the  Naval  Training 
Equipment  Center  provided  assurance  that  these  and 
other  discrepancies  could  and  would  be  corrected 
prior  to  the  start  of  T*e2  data  collection. 

Having  assurance  that  the  2B35  could  be  made 
to  perform  properly,  it  was  then  necessary  to 
Identify  the  training  stages  for  which  suitable 
Image-generating  data  bases  were  available.  The 
data  bases  for  ongoing  FAM  and  WEP  training  were  of 
obvious  potential  use.  In  addition,  although  they 
were  not  being  used  In  training,  data  bases  for 
Night  Familiarization  (NF)  and  Field  Carrier 
Landing  Practice/Carrier  Qualification  (FCLP/CQ) 
were  judged  to  be  of  potential  use.  There  were, 
thus,  four  training  stages,  FAM,  NF,  WEP,  and 
FCLP/CQ,  *d)1ch  could  be  examined  during  the  study 
without  extensive  new  data  base  development. 

Evaluation  Design 

The  first  evaluation  design  consideration  was 
the  selection  of  a  basic  training  strategy  that 
could  be  effected  within  ongoing  Navy  UPT.  The 


second  was  the  technical  concerns  underlying 
development  of  an  experimental  design  as  such. 

Training  Strategies.  Two  training  strategies 
were  entertained.  One  would  utilize  an  Individual, 
traln-to-proflclency  training  regimen,  while  the 
other  would  involve  a  lock-step,  constant- time 
training  strategy.  The  first  approach  had  the 
advantage  of  providing  Information  about  time  or 
trials  required  for  training  and  would  allow  direct 
computation  of  both  transfer  ratios  (TRs)  and 
transfer  effectiveness  ratios  (TERs).  Unfortu¬ 
nately,  this  approach  could  not  be  Implemented 
without  making  major  changes  to  the  Navy  training 
syllabus  and  established  scheduling  procedures, 
changes  that  the  Navy  could  not  support.  Since  the 
fixed-time  training  regimen  was  also  the  training 
regimen  in  being.  It  was  selected  for  the  T?E2. 
The  disadvantages  of  this  regimen  from  the  research 
viewpoint  were  more  than  offset  by  Its  operational 
suitability. 

Experimental  Design.  The  experimental  design 
was  basical ly  a  three  group,  repeated  measures 
design  for  the  FAM,  NF,  and  FCLP/CQ  stages.  It 
provided  for  three  levels  of  the  simulator  time 
variable  at  each  of  these  stages.  At  the  WEP 
stage,  however,  because  of  limitations  on  sample 
size,  the  design  called  for  only  two  major  groups, 
a  simulator  group  and  a  nonsimulator  control  group. 

Training  Treatments.  The  2B35  training 
treatments  were  administered  in  the  following 
manner.  Each  student  scheduled  to  begin  the  FAM 
Stage,  was  randomly  assigned  to  one  of  three  FAM 
treatment  groups:  Group  A,  which  received  four 
2-hour  sessions  in  the  2B35  followed  by  five 
flights  In  the  TA-4J ;  Group  B,  Oiich  received  two 
2-hour  2B35  sessions  prior  to  its  five  training 
flights  In  the  TA-4J ;  and  Group  C,  the  control 
group,  which  flew  seven  flights  in  the  TA-4J  and 
received  no  2B35  training.  This  assignment 
procedure  continued  until  each  group  had  twenty 
subjects. 

As  the  three  FAM  groups  approached  WEP 
training,  half  of  each  group  was  randomly  assigned 
to  a  WEP  A  simulator  group,  vrfiile  the  other  half 
was  assigned  to  a  WEP  C  nonsimulator  control  group. 
This  allowed  examination  of  FAM-WEP  interaction 
effects.  The  WEP  A  group  received  four  2-hour  2B35 
training  sessions,  while  the  WEP  C  group,  of 
course,  received  none.  Each  group  C N=30 )  then 
received  seven  WEP  training  flights  In  the 
aircraft. 

For  FCLP/CQ  training,  the  original  A,  B,  and  C 
groups  were  reformed,  with  the  A  group  receiving 
three  1-hour  2B35  sessions  and  the  B  group  one  such 
session.  The  C  group,  of  course  received  no  2B35 
training  other  than  the  brief  carrier  deck 
emergency  procedures  training  session  received  by 
all  students  during  CQ  training. 

Performance  Measurement 

The  existing  Navy  UPT  grading  system  evaluates 
student  pilot  performance  on  a  four-point  scale: 
Unsat1sfactory=l;  Below  Average*2;  Average-3;  and 
Above  Averages.  This  grading  system  was  quickly 
discounted  as  useful  for  study  purposes  due  to  Its 
lack  of  discrimination  sensitivity.  For  example, 
more  than  90%  of  the  maneuver  grades  given  were 
"average."  Such  “normative"  grades  may  be  adequate 
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for  local  management  of  students'  dally  progress, 
but  they  were  judged  not  sufficiently  sensitive  to 
be  of  use  In  training  program/treatment  effects 
evaluations. 

As  a  consequence,  a  performance  measurement 
system  involving  Instructor  recordings  of  observed 
student  control  over  specified  maneuver-critical 
aircraft  parameters  such  as  heading,  altitude, 
time,  and  airspeed  was  developed  for  T2E2  use. 
Since  this  measurement  approach  has  been  found  by 
many  Investigators  to  provide  reliable  and 
discriminating  data  In  flight  training  studies.  It 
was  deemed  appropriate  for  this  effort. 

Separate  performance  recording  booklets  were 
developed  for  FAM,  NF,  and  WEP.  As  will  be 
explained  shortly,  FCLP/CQ  performance  was  measured 
differently.  For  the  FAM,  NF,  and  HEP  maneuvers, 
the  Instructor  was  required  to  observe  and  record 
at  selected  times/places  within  a  maneuver  whether 
the  student  was  within  prescribed  tolerances  for 
specific  parameters.  Some  parameters  were  measured 
only  once  during  a  maneuver,  while  others  were 
sampled  two  or  more  times.  Figure  1  illustrates 
the  procedure.  In  the  example  shown,  the 
Instructor  noted  Takeoff  rotation  airspeed  to  be 
5-10  knots  high,  attitude  as  proper,  and  rate  as 
being  over-controlled. 


TAKEOFF  ROTATION 


An 


RATE 


U/C  /\  f 


Figure  I.  Sample  Segnent  of  a  FAM 

Performance  Recording  Booklet. 


Performance  recordings  were  converted  to  error 
scores  by  considering  all  marks  within  the  center 
triangles  or  squares  “correct.”  Any  other  marks, 
left  or  right,  counted  as  errors.  By  summing 
errors  across  all  Items  In  the  maneuver,  a  maneuver 
total  error  score  was  obtained;  by  sunning  across 
parameters  measured  repeatedly,  a  parameter-control 
error  score  was  derived.  These  maneuver  and 
parameter  scores  were  the  primary  data  analyzed  for 
the  FAM  and  NF  stages. 


student  Is,  however,  under  the  radio  control  and 
guidance  of  the  Landing  Signal  Officer  (LSO)  on  all 
FCLP/CQ  flights.  Further,  the  LSO  routinely 
observes  each  student's  pattern  and  landing— In 
particular  from  final  turn  to  touchdovm— and 
records  descriptive  Information  about  each  pass 
using  a  standardized  error  rotational  shorthand.  A 
procedure  was  developed  thereby  these  data  were 
routinely  transcribed  to  Navy  Landing  Trend 
Analysis  forms  and  subsequently  converted  to  error 
scores  for  each  student  landing  on  each  flight. 
Error  scores  thus  derived  were  the  error  data 
employed  for  FCLP/CQ  analysis.  In  addition,  the 
LSO's  subjective  grades  were  also  available  for 
each  of  the  14  FCLP/CQ  Flights,  and  these  were 
analyzed  separately. 

Subjects 

Students  enter  the  Advanced  Jet  Phase  of  UPT 
In  relatively  small  weekly  or  bi-weekly  Increments 
of  from  four  to  six  per  "class."  Thus,  to  achieve 
the  desired  N  of  at  least  20  subjects  per  FAM 
treatment  group,  it  was  necessary  to  use  In  the 
T2E2  all  students  who  entered  this  training  phase 
at  Chase  Field  from  June  through  September  1978. 
In  all,  a  total  of  64  students  were  Involved.  Of 
these,  59  were  subsequently  graduated  as  Naval 
Aviators. 


Implementation 


A  nunber  of  special  Implementing  procedures 
were  required.  Foremost  among  these  were  dally 
lesson  guides  which  controlled  the  sequence  and 
content  of  device  Instruction  and  defined  the 
procedures  to  be  followed  during  data  collection. 
Second  In  Importance  were  the  Instructor/manager 
training  sessions  conducted  to  assure  standardized 
device  training  and  recording  booklet  use.  Last, 
but  not  least,  was  the  close  surveillance 
maintained  by  the  research  team  over  all 
T2EZ  activities  throughout  the  protracted 
training/data  collection  period  which  extended  fro- 
June  1978  into  March  1979. 


HI.  RESULTS 

The  TZE^  study  had  three  general  obj  '  ^5: 
(1)  to  determine  whether  Device  2B3F  v  jdl 
training  produced  demonstrable  learning;  (2)  to 
discover  whether  such  learning  as  was  found 
transferred  to  the  TA-4J  aircraft;  and  (3)  to 
Identify  those  syllabus  stages  wherein  35 
training  would  be  of  greatest  utility  assuming  int 
meaningful  transfer  were  obtained. 


Learning  In  the  2B35 


There  was  an  additional  source  of  data  for  the 
HEP  stage.  Hhlle  the  recording  booklet  was  used  to 
provide  error  scores  for  HEP  pattern  flying  from 
the  abeam  position  to  the  release  point  during  30° 
dive-angle  practice  bombing,  clock  angle  and  miss- 
distance  scores  were  also  obtained  for  six  practice 
bomb  drops  on  each  training  flight-device  and 
aircraft.  These  bomb  drop  data  thus  provided  a 
second  data  source. 

The  booklet  recording  procedure  was  not 
suitable  for  FCLP/CQ  performance  measurement,  since 
an  Instructor  Is  not  on  board  the  aircraft.  The 


The  FAM  and  HEP  stages  were  the  ones  In  which 
the  device- trained  group  (Group  A)  received  four 
consecutive  2-hour  training  sessions,  thus  allowing 
the  clearest  examination  of  across-trlal  learning. 
Using  group  percent  error  scores  for  each  maneuver 
flown,  learning  curves  were  plotted  for  the  four 
FAM  maneuvers  and  for  the  HEP  pattern  data  and  bomb 
scores.  All  five  curves  reflected  a  substantial 
reduction  In  group  percent  error  scores  from  the 
first  to  last  trial.  To  Illustrate,  Figure  2 
presents  the  Group  A  learning  curves  for  the  FAM 
Stage  Full  Flap  Landing  and  the  HEP  pattern  over 
the  four  simulator  sessions. 
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Figure  2.  Percent  Error  on  FAM  Full  Flap 
Landing  and  HEP  Pattern  by 
Simulator  Session  (Group  A) 


From  these  curves,  it  is  apparent  that 
learning  did  occur  In  the  2835.  The  reduction  in 
errors  by  the  A  Group  was  statistically  significant 
for  all  of  the  FAM  maneuvers  but  the  Barrel  Roll. 
For  the  WEP  training,  even  though  the  reduction  in 
errors  from  the  first  to  the  fourth  simulator 
sessions  was  not  statistically  significant,  the 
trend  for  that  maneuver  was  also  for  reduction  In 
errors.  Also,  that  the  HEP  Pattern  data  did  not 
asymptote,  as  was  typical  of  the  FAM  curves, 
suggests  that  further  WEP  device  training  could  be 
productive. 

Since  NF  and  FCLP/CQ  device  training  was 
limited  to  two  sessions  for  the  A  Group  and  only 
one  session  for  the  B  Group,  presentation  of 
learning  curves  for  NF  is  not  useful.  However,  It 
should  be  noted  that  the  Iff  A  Group  did  show  a 
statistically  significant  reduction  In  percent 
error  score  over  the  two  device  sessions  for  the 
Full  Flap  Landing,  further  evidence  of  learning  In 
the  2B35. 

During  FCLP/CQ  device  training,  both  day  and 
night  Chase  Field  FCLP  scenes  were  used  for  Groups 
A  and  B,  but  the  carrier  scene  was  used  only  during 
the  A  Group's  training.  As  a  consequence,  device 
learning  In  this  stage  could  be  examined  only  by 
comparing  Group  A's  night  landing  performance 
during  their  first  and  second  2B35  sessions.  This 
analysis  showed  no  significant  difference  over 
trials  for  the  error  scores  derived  from  the 
Landing  Trend  Analysis  forms.  There  was,  however, 
significant  improvement  In  LSO  subjective  grades 
over  the  two  sessions,  thus  providing  some 
Indication  that  FCLP/CQ  learning  resulted  from 
practice  In  the  2B35. 

In  summary.  It  appears  that  the  tvpes  of 
visual  tasks  and  maneuvers  used  In  the  TZE*  effort 
can  be  learned  In  the  2B35.  While  there  were 
differences  In  learning  rates  and  In  absolute 
performance  levels  achieved  across  these  maneuvers, 
there  remains  little  doubt  that  the  device  can  be 
used  to  learn  visually  cued  maneuvers.  The 
critical  Issue  Is,  of  course,  Aether  learning  In 


the  2B35  influences  subsequent  student  performance 
In  the  1A-4J  aircraft. 

Learning  In  the  Aircraft 

The  objective  aircraft  data  for  FAM,  NF,  WEP, 
and  FCLP/CQ  were  analyzed  as  follows.  The  data 
from  the  FAM,  NF,  and  WEP  booklets  and  the  derived 
error  scores  from  the  FCLP/CQ  Landing  Trend 
Analysis  forms  were  analyzed  on  the  Unlvac  1100 
computer  at  the  Arizona  State  University  Computer 
Center.  The  program  "Mul tl variance"  was  used  for 
the  univariate  and  multivariate  analyses  of 
variance,  covariance,  and  regression.  Supplemen¬ 
tary  analyses  of  these  data  were  also  performed  on 
the  Unlvac  1100  using  programs  from  the  Statistical 
Package  for  the  Social  Sciences.  The  .05 
probability  level  was  selected  as  the  significance 
point  for  all  analyses. 

The  WEP  practice  bomb  score  data  were  examined 
separately  using  univariate  analyses  and  correlated 
^  tests.  These  WEP  data  were  analyzed  first  as 
circular  error  data  and  then,  by  trigonometric 
conversion,  as  the  vertical  and  horizontal 
components  of  the  circular  error. 

The  LSO  subjectlves  grades  available  from  the 
FCLP/CQ  flights  were  also  analyzed  by  either 
univariate  analyses  or  correlated  t^  tests,  as 
appropriate. 

FAM  Stage.  The  fact  that  the  no-device  time 
control  group  (Group  C)  received  two  more  aircraft 
training  flights  than  did  Groups  A  and  B  allowed 
for  two  types  of  comparisons.  Comparison  one,  the 
traditional  transfer  approach,  compared  the  three 
FAM  groups  on  their  first,  second,  third,  fourth, 
and  checkrlde  performances;  this  omitted  the  C 
Group's  fifth  and  sixth  training  flights.  In 
comparison  two,  the  C  Group’s  first  and  second 
flights  were  Ignored,  and  inter-group  comparisons 
were  made  between  the  A  and  B  Groups'  first  flights 
against  the  C  Group's  third,  the  A/B  second  with 
C's  fourth,  and  so  on,  up  to  the  checkrlde.  These 
arrangements  are  shown  In  Table  1. 


TABLE  1 


Device  2B35  versus  Control  Group  Comparison 
Paradigms  (FAM) 

Group  Flights 


Comparison 

A/B 

1  2 

3 

4 

Checkrlde 

1 

C 

1  2 

3 

4 

(5)(6) 

Checkrlde 

Comparison 

A/B 

1 

2 

3  4 

Checkrlde 

2 

C 

(1)U) 

3 

4 

5  6 

Checkrlde 

Multivariate  analyses  (MANOVAs)  for  both  FAM 
comparison  arrangements  yielded  nonsignificant  MVFs 
for  treatments,  but  the  MVFs  across  flights  were 
significant  for  all  maneuvers  except  the 
Straight-In  Precautionary  Approach  (MVF*2.16; 
df -20. 37 ;  £< .03) .  While  the  nonsimulator  C  Group 
tended  to  make  fewer  errors  than  the  two  simulator 
groups  (A  4  B),  the  performance  of  the  control 
group  across  the  seven  aircraft  training  flights 
was  not  significantly  different  from  the 
performance  of  the  simulator  groups  which  received 
only  five  aircraft  training  flights. 


These  results  show,  as  would  be  expected,  that 
all  three  groups  Improved  with  practice.  However, 
the  results  do  not  lend  any  substantial  support  to 
a  significant  simulator- to-alrcraft  transfer 
effect,  other  than  the  fact  that  the  simulator 
groups  which  received  two  fewer  aircraft  flights 
performed  In  a  fashion  not  significantly  different 
from  that  of  the  control  group.  On  the  other  hand, 
neither  do  the  results  lend  support  to  the  converse 
Inference  that  the  all-aircraft  training  regimen 
was  superior  to  the  simulator-plus-aircraft 
regimen. 

Unfortunately,  subject  limitations  precluded 
having  a  fourth  group  which  could  have  received 
only  five  aircraft  flights  and  no  2B35  training. 
Such  a  group  would  have  provided  more  conclusive 
evidence  regarding  the  degree  to  which  transfer 
from  the  2B35  did  or  did  not  account  for  the  A/B 
Group  performance.  Without  such  a  group,  the 
possibility  remains  that  the  FAM  checkrlde  can  be 
"passed"  as  well  with  only  five  TA-4J  flights  and 
no  device  time  as  It  Is  with  seven  aircraft  rides. 

Night  FAM  Stage.  The  NF  error  data  from  the 
three  treatment  groups  yielded  a  significant  MVF 
value  favoring  Group  C  (MVF=2.84;  df=6,  110; 

£<.02).  In  view  of  the  cl  early  better  performance 
of  the  nonsimulator  Group  C,  use  of  the  2B35  for  NF 
training  cannot  be  supported. 

FAM  Ball -Throttle  Control.  As  indicated 
earlier,  there  were  significant  concerns  over  the 
power  response  and  Fresnel  Lens  Optical  Landing 
System  (FLOLS)  bal 1 -tracking  characteristics  of  the 
2B35.  Since  substantial  software  and  engineering 
changes  to  the  2B35  had  been  made  to  support  the 
T2e2  effort.  It  was  of  special  Interest  to 
determine  whether  the  device  might  have  negative 
transfer  characteristics  In  these  areas.  The  many 
power  control  and  ball-tracking  measures  which  had 
been  obtained  across  the  several  FAM  and  NF  Full 
Flap  Landing  maneuvers  flown  during  the  aircraft 
training  flights  allowed  an  examination  of  both 
treatment  and  practice  effects  of  device  training 
on  these  critical  parameters. 

Multivariate  analysis  of  the  data  from  the  FAM 
stage  sho**d  no  significant  treatment  effects.  A 
similar  analysis  of  the  NF  data  Indicated  a 
significant  treatment  effect  In  favor  of  the  A 
simulator  group  (MVF=3.00;  df=12,  104;  p<.01).  The 
evidence  from  these  two  analyses  slightly  favors 
the  trainer  groups  and,  at  the  very  least,  supports 
a  conclusion  that  subjects  with  2B35  training  time 
clearly  were  not  inferior  to  their  all-aircraft 
peers  with  respect  to  ball  control  and  power 
control.  In  view  of  these  outcomes,  there  is  no 
reason  to  believe  that  the  2B35  training  given  A 
and  B  Groups  resulted  In  any  negative  transfer  In 
these  task  areas. 

WEP  Stage  Error  Data.  The  MANOVA  of  the  WEP 
pattern  error  data  yielded  a  significant  treatment 
effect  in  favor  of  the  simulator  group  (MVF=2.63; 
df=10,  47;  p< . 02 ) .  This  was  the  most  clear-cut 
area  of  simulator  advantage  in  the  study.  The  data 
also  yielded,  as  expected,  a  significant  effect 
across  flights,  i.e.,  a  learning  or  practice  effect 
(MVF *2. 75;  df=20,  37;  £<.01). 

The  analyses  of  the  error  scores  also  showed  a 
significant  interaction  effect  between  the  two  WEP 
treatments  (A  or  C)  and  previous  FAM/NF  treatments 


(A,  B,  or  C)  (MVF=1 .65;  df=40,  74;  £<.03).  The 
pattern  of  this  interaction  suggests  that,  while 
WEP  A  treatment  positively  Influenced  WEP  pattern 
performance  overall,  having  had  the  previous  2B35 
FAM/NF  Group  A  training  was  slightly 
disadvantageous  to  later  WEP  flight  performance, 
FAM/NF  Group  B  membership  had  no  clear  effect  on 
WEP  performance,  and  FAM/NF  Group  C  membership  was 
slightly  advantageous  to  later  WEP  performance. 
One  may  speculate  that  students  and  instructors  who 
had  already  spent  substantial  time  in  the  2B35 
(FAM/NF  Group  A)  may  have  approached  further 
exposure  to  the  device  for  WEP  training  less 
positively  than  those  being  Introduced  to  it  for 
the  first  time. 

WEP  Stage  Practice  Bomb  Scores.  As  previously 
described,  practice  bomb  scores  were  first  analyzed 
in  terms  of  mean  miss  distance  and  then  In  terms  of 
the  mean  horizontal  and  vertical  error  components 
of  that  miss  distance.  Univariate  analyses  of 
these  three  types  of  miss  distances  showed  no 
significant  treatment  effects,  though  the  A  Group's 
miss  distances  tended  to  be  somewhat  less  than 
those  of  the  control  group  (Group  C).  The 
advantage  of  2B35  training  to  the  A  Group  that  was 
apparent  from  the  WEP  objective  pattern  data 
analysis  did  not  significantly  Influence  their 
bombing  accuracy,  though  the  trend  was  for  them  to 
be  more  accurate. 

Practice  effects  as  shown  by  these  bomb  scores 
were  Interesting.  While  there  was  substantial 
Improvement  In  overall  miss  distance  over  trials, 
the  horizontal  error  component  Improved  relatively 
little.  The  bulk  of  the  improvement  was  In  the 
vertical  error  components.  Also,  horizontal  miss 
distances  were  significantly  smaller  than  the 
vertical  miss  distances.  These  data  suggest  that 
bomb-run  lineup  Is  not  a  difficult  skill,  and 
consequently,  there  was  little  room  for  Improvement 
in  horizontal  accuracy.  On  the  other  hand,  the 
relatively  greater  vertical  error  component  is 
related  to  the  pattern  elements  such  as  dive  angle 
and  release  altitude  and  seems  to  represent  the 
more  difficult  aspect  of  the  bombing  task. 

FCLP/CQ  Stage.  The  objective  FCLP/CQ  error 
data  were  subjected  to  MANOVA  analysis,  and  the 
subjective  grades  were  examined  by  univariate 
analysis.  Multivariate  analysis  of  the  LSO-derlved 
error  data  showed  no  significant  effects  due  to 
treatments,  I.e.,  there  was  no  evidence  of  any 
differences  between  simulator  and  nonsimulator 
training  groups  In  the  aircraft  portion  of  FCLP/CQ. 
Further,  there  was  no  significant  change  In 
performance  from  early  FCLP  flights  to  the  later 
flights,  I.e.,  these  LSO  error  data  provided  no 
evidence  of  FCLP/CQ  skills  learning  In  the 
aircraft. 

While  It  Is  possible  that  there  was  no  change 
In  student  performance  over  the  13  FCLP  flights, 
I.e.,  no  learning,  It  Is  more  likely  that  the  LSO 
error  data  reflect  a  shifting  frame  of  reference  as 
to  vrfiat  constitutes  an  error.  This  lack  of 
discrimination  In  the  LSO  error  data  was 
disappointing  In  view  of  the  usefulness  of  such 
data  that  has  been  reported  by  Brlctson  and  Burger 
(3)  In  their  evaluation  of  the  A-7E  Night  Carrier 
Landing  Trainer  (NCLT). 

As  has  been  noted,  the  LSO  subjective  grade 
data  vere  also  available  for  FCLP/CQ  flights. 
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These  grades  Mere  subjected  to  univariate  analyses 
of  variance  which  also  showed  no  significant 
treatment  effects.  However,  all  three  groups  made 
significant  gains  from  their  first  three  FCLP 
flights  to  their  last  three  (A  Group:  t=2.51, 
df=14,  p< .01 ;  B  Group:  t*5.08,  df=17,  p<.01;  C 
TTroup:  ^=5.20,  df=19,  j><.0T). 

The  findings  of  a  clear  practice  effect 
reflected  by  LSO  grades  is  consonant  with  the 
preceding  statement  concerning  the  likelihood  of  a 
shifting  scale  in  the  LSO-derlved  error  scores. 

There  were  a  number  of  problems  other  than 
measurement  that  further  weakened  the  FCLP/CQ 
portion  of  the  study.  Among  these  were  schedule 
conflicts  that  unavoidably  disrupted  the  orderly 
flow  of  2B35  training,  missing  data,  etc.  For 
these  reasons,  this  evaluation  of  the  2B35  for  use 
in  FCIP/CQ  must  be  viewed  as  inconclusive. 


IV.  DISCUSSION 

The  central  purpose  of  this  study  was  to 
conduct  an  evaluation  of  the  training  transfer 
effects  of  Device  2B35  for  the  visually  cued  tasks 
required  in  t'-e  Navy's  Advanced  Jet  Undergraduate 
Pilot  Training.  The  design  chosen  for  the 
evaluation  was  necessarily  one  feasible  of 
implementation  within  ongoing  operational  training 
that  would  provide  the  Navy  with  operationally 
meaningful  transfer  information. 

The  results  of  this  study  are  generally 
supportive  of  visual  simulation,  though  not 
strongly  so,  and  they  are  consistent  with  the 
findings  from  similar  efforts  in  which  evidence  of 
positive  transfer  has  been  obtained,  but  in  which 
the  effects  have  been  modest. 

Two  recent  Air  Force  UPT  studies  are  of 
interest  in  this  regard.  In  the  first  (4), 
transfer  of  basic  contact  skills  from  visual 
simulator  training  to  the  T-37  primary  jet  training 
aircraft  was  examined,  while  the  second  (5) 
examined  transfer  for  aerobatic  maneuvers.  In  both 
studies,  evidence  of  positive  transfer  was 
obtained,  but  the  effects  were  not  dramatic.  In 
addition  to  these  two  efforts  dealing  with  UPT 
training,  there  have  been  several  other  research 
studies  addressing  the  transfer  of  training 
potential  of  visual  simulators  for  basic  fighter 
maneuvers,  aerobatics,  transition  skills,  and 
weapons  delivery.  For  example,  one  such  study  (6) 
conducted  in  a  well -controlled  experimental 
situation,  found  a  consistent  trend  toward  positive 
training  transfer  for  Navy  F-4  pilots  on  basic 
fighter  maneuvering  tasks--but,  except  for  one 
maneuver,  none  of  the  effects  was  large  enough  to 
be  statistically  significant.  Similar  results  were 
obtained  by  the  Air  Force  during  their  evaluation 
(7)  of  the  Simulator  for  Air-to-AIr  Combat  (SAAC); 
a  small  positive  trend,  but  not  of  statistical 
significance.  In  contrast,  recent  studies  of  the 
transfer  effects  of  visual  simulator  training  for 
weapons  delivery  appear  to  have  the  most  impressive 
results.  One  in  particular  (8)  found  substantial 
positive  transfer  for  air-to-surface  weapons 
del ivery. 

As  can  be  seen  from  the  several  studies  cited, 
the  use  of  visual  simulation  has  produced 
moderately  positive  results  for  transition/ 


famil iarization  type  skills  and  some*rfrat  stronger 
positive  results  for  weapons  delivery  skills. 
Thus,  the  results  of  this  T^E?  can  be  interpreted 
as  providing  modest  additional  support  to  the  use 
of  visual  simulation  for  such  military  piloting 
skills. 

In  contrast,  while  the  A-7E  NCLT  study 
previously  cited  (3)  found  a  significant  transfer 
effect  from  that  trainer  to  night  carrier 
qualification,  no  such  effect  was  found  here. 
Whether  the  divergence  between  that  study's 
findings  and  the  present  results  reflects 
differences  in  measurement  sensitivity,  differences 
in  the  way  the  device  was  used,  or  differences 
between  day  and  night  CQ  cannot  be  determined  here. 
However,  the  NCLT  results  do  suggest  FCLP/CQ  as  an 
area  that  might  be  worthy  of  further  investigation 
in  the  UPT  setting. 

Findings 

Learning  in  the  2B35.  The  results  show 
clearly  that  students  learned  in  the  2B35.  While 
some  of  the  maneuvers  showed  greater  learning  than 
did  others,  virtually  all  reflected  improvement  in 
task  performance  as  a  function  of  practice.  It  is 
apparent  also  that  some  of  the  device  training 
tasks  were  relatively  easy,  while  others  were 
relatively  more  difficult.  Since  there  is  only  one 
2B35  device  at  each  Navy  Advanced  Jet  training 
site,  this  finding  has  implications  for  device  use. 
Tasks  easy  to  learn  in  the  device  should  not  be 
given  much  emphasis,  particularly  if  the  device 
time  so  consumed  prevents  time  being  spent  on 
harder  to  learn  tasks. 

The  most  interesting  2B35  learning  result, 
perhaps,  is  with  reference  to  WEP  training.  For 
both  the  flight  pattern  skills  and  for  bombing  miss 
distance,  it  is  clear  that  asymptote  had  not  been 
reached  at  the  end  of  four  trainer  periods.  This, 
in  combination  with  the  positive  transfer  evidence 
and  the  fact  that  there  is  still  room  for 
considerable  improvement  in  the  inflight  bombing 
skills,  suggests  that  additional  2B35  WEP  training 
beyond  the  four  periods  might  be  beneficial. 

Transfer  to  the  Aircraft.  The  transfer 
results  provide  general  support  for  the  continued 
use  of  the  2B35  in  Navy  Advanced  Jet  training,  but 
its  utility  in  the  different  stages  of  that 
training  varies.  Specific  recommendations 
concerning  the  various  stages  are  as  follows. 

FAM  stage.  Overall,  the  transfer  data 
from  the  present  study  neither  strongly  support  nor 
refute  the  use  of  the  2B35  for  the  various  FAM 
stage  maneuvers.  The  all-aircraft  control  group 
did  not  show  any  significant  flight  advantage  over 
the  device-trained  groups,  but  neither  do  the  data 
support  the  contention  that  the  device  produced 
negative  transfer  with  reference  to  the  critical 
flight  skills  of  ball  control  and  power  control. 
The  finding  that  device-trained  students  achieve  in 
five  aircraft  flights  a  skill  level  that  appears  to 
be  the  equivalent  of  that  achieved  by  control 
students  in  seven  aircraft  flights  can  be  construed 
as  supportive  of  continued  use  of  the  2B35  for  FAM 
instruction,  but  no  clear  cut  performance  advantage 
was  shown  for  device-trained  students.  A  saving  of 
two  aircraft  flights  is  a  saving  of  some 
consequence,  but  it  is  possible  that  students  who 
received  neither  the  two  extra  flights  nor  the  2B35 
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training  might  perform  equally  well.  Such  a 
determination,  however,  would  have  required  a 
second  control  group,  a  requirement  beyond  study 
resources. 

Based  on  all  these  considerations,  It  is 
concluded  that  continued  use  of  the  2B35  for  FAM 
training  Is  warranted. 

Night  FAM  stage.  In  view  of  the  apparent 
lack  of  difficulty  with  the  NF  maneuvers,  and  since 
the  control  group  showed  some  f)  Ight  advantage  over 
the  trainer  groups,  It  is  concluded  that  the  2B35 
will  not  provide  significant  training  benefit  in 
the  NF  stage  of  Advanced  Jet  training. 

WEP  stage.  The  resul ts  of  the 
T2E2  effort  with  reference  to  2B35  In  WEP  training 
provide  relatively  strong  support  for  continued  use 
of  the  device  in  this  stage.  The  acquisition  of 
WEP  flight  pattern  skills  In  the  aircraft  Is 
clearly  enhanced  by  2B35  training.  It  Is  of  some 
interest  to  note,  though,  that  the  improvement  in 
WEP  aircraft  flight  pattern  skills  that  results 
from  the  2B35  training  Is  not  accompanied  by 
statistically  significant  differences  In  bomb 
scores,  though  the  differences  did  favor  the 
trainer  group. 

The  data  concerning  vertical  and 
horizontal  error  components  of  bomb  scores  are  of 
Interest  in  terms  of  Instructional  emphasis.  This 
finding.  In  combination  with  the  fact  that 
asymptotic  performance  level  was  not  reached  In 
either  the  trainer  or  the  aircraft,  suggests  that 
further  2B35  WEP  training  might  be  beneficial, 
particularly  to  emphasize  the  obvious  relevance  of 
dive  angle  and  release  to  the  vertical  error 
component. 

FCLP/Cq  stage.  In  view  of  problems 
experienced  In  the  FCLP/CQ  stage  of  the  study  and 
the  lack  of  adequate  data,  no  firm  conclusion  Is 
drawn  relative  to  the  use  of  the  2B35  to  support 
FCLP/CQ  stage  training.  On  an  analytical  basis, 
the  device  would  seem  to  have  potential  for  such 
use,  but  on  the  basis  of  the  empirical  results  of 
this  study,  such  use  can  neither  be  endorsed  nor 
rejected. 

Implications  for  the  Future 

This  effort  adds  support  to  the  growing  body 
of  literature  that  shows  simulators  can  provide  a 
positive  contribution  to  the  meeting  of  many  visual 
training  requirements,  in  particular  In  the  areas 
of  contact  transition  or  familiarization  training 
and  visual  weapons  delivery.  However,  It  also 
Indicates  that  use  of  visual  devices  should  be 
based  on  a  careful  analysis  of  the  task  training 
requirements,  the  device's  capabilities,  and  the 
training  system  In  which  It  will  be  employed. 

The  utility  of  any  given  visual  device  must  be 
viewed  In  this  systems  context.  For  example, 
because  of  the  Navy's  use  of  the  "Instruments 
first,  contact  later"  syllabus  sequence  in  Advanced 
Jet  training,  the  student  has  considerable  skill  in 
flying  the  TA-4J  aircraft  on  Instruments  before  he 
Is  Introduced  to  the  2B35  visual  simulator.  This 
Instructional  sequence  probably  limits  somewhat  the 
2B35's  potential  contribution  to  the  acquisition  of 
FAM  aircraft  skills.  An  alternative  sequencing  or 
use  of  a  visual  device  In  an  earlier  phase  of 


training  (e.g.,  the  T-2  Basic  Jet  Phase)  might 
result  In  a  different  potential.  Future  orograms, 
such  as  the  VTXTS,  must  examine  sue.,  factors 
carefully  If  visual  simulation  and  simulation  In 
general  are  to  contribute  both  effectiveness  and 
efficiency  to  undergraduate  training. 

Though  not  specifically  detailed  In  this 
paper,  three  general  requirements  for  effective 
future  simulator  training  were  highlighted  In  the 
T2E2  effort  and  are  worth  noting.  These  are  the 
requirements  for  (1)  adequate  maintenance  and 
personnel  support,  (2)  more  objective  measures  of 
performance,  and  (3)  Instructor  training  In  the 
Instructional  use  of  simulation.  With  adequate 
attention  to  these  requirements  and  to  the  device 
and  training  system  factors  noted,  visual 
simulation  offers  significant  potential  for 
Improving  future  UPT  programs. 
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ABSTRACT 


Since  January,  1978,  United  Airlines  has  been  using  the  PLATO  computerized  system  to 
manage  an  individualized,  criterion  referenced  instructional  (CRI)  program  for  newly  hired 
pilots.  This  program  was  designed  utilizing  the  CRI  concept  to  take  advantage  of  trainees' 
existing  knowledge  repertoires,  the  different  rates  at  which  trainees  learn,  and  for 
scheduling  flexibility.  Criterion  tests,  used  to  measure  expected  performance,  are 
presented  on  the  PLATO  Computer  Managed  Instruction  (CMI)  system;  learning  resources  are 
printed  materials,  slide-tapes  and  video-tapes.  The  next  program  that  was  developed  and 
implemented  in  January,  1980,  Initial  First  Officer,  utilizes  both  the  CMI  and  Computer 
Assisted  Instruction  (CAI)  features  of  PLATO.  Also  based  on  CRI  design,  this  program  is 
presented  on  PLATO  as  CAI  lessons  yet  still  contains  the  advantages  of  CMI  record  keeping. 
In  addition,  no  external  media  is  necessary,  the  trainee  is  provided  immediate  feedback,  and 
appropriate  learning  material  is  given  on  the  spot. 


BACKGROUND 


The  United  Airlines  Flight  Operations 
Training  Center,  located  in  Denver,  Colorado,  is 
a  central  facility  responsible  for  all  training 
of  United's  5,500  pilots.  The  largest  activity 
is  Transition  training  which  qualifies  pilots 
when  they  move  from  one  airplane  to  another.  In 
addition,  semi-annual  Proficiency  checks. 
Recurrent  training.  Requalification  training. 
Emergency  Procedures  training.  New  Hire  training, 
Initial  Captain  and  Initial  First  Officer 
training  is  conducted  continuously. 


INTRODUCTION  TO  THE  NEW  HIRE  COURSE 


In  1976,  the  Flight  Training  Program 
Development  group  at  United  was  given  the 
responsibility  to  develop  a  training  program  for 
newly  hired  pilots  (Second  Officers). 
Specifically  it  was  essential  that  the  training 
would  assure  that  any  newly  hired  pilot  would  be 
able  to  enter  any  of  the  regular  transition 
training  programs  prepared  to  meet  the  objectives 
of  those  programs  with  no  modifications  to  the 
existing  transition  training.  This  meant  that 
new  pilots  had  to  be  brought  up  to  the  entry 
level  of  knowledge  on  which  the  transition 
training  programs  are  based.  To  compound  this 
problem,  the  expectations  of  the  availability  of 
pilot  candidates  and  the  qualification  standards 
for  selection  led  us  to  expect  a  wide  range  of 
student  experience  in  each  class.  We  could 
expect  students  with  experience  ranging  from 
light  airplane  time  to  those  with  extensive 
multi-engine  jet  experience  from  the  military. 


It  had  been  eight  years  since  United  had  hired 
new  pilots,  and  they  had  been  trained  with  a 
conventional,  instructor-led  classroom  course. 
The  factors  facing  us  now,  however,  pointed  out 
the  inefficiency  of  a  traditional,  lock-step 
program  which  assumes  that  all  trainees  require 
the  same  training.  A  program  which  requires  the 
experienced  pilot  to  sit  in  a  classroom  receiving 
the  same  lectures  as  the  minimally  qualified 
pilot  is  not  only  wasteful  and  inefficient,  but 
boring  as  wel 1 . 


NEW  HIRE  COURSE  DESIGN 


Based  on  an  analysis  of  the  situation,  then, 
it  was  determined  that  utilizing  the  principles 
of  Criterion  Referenced  Instruction  (CRI)  would 
best  meet  the  need.  This  meant  that  the  program 
would  be  results  oriented  and  tailored  to  meet 
individual  needs.  Criterion  tests  would  be  used 
to  measure  expected  performance  with  mastery  of 
all  the  tests  necessary  to  complete  the  course. 


The  acceptance  of  CRI  principles  allows  the 
content  of  the  training  to  be  as  much  or  as 
little  as  a  particular  trainee  needs  to  help 
master  the  criterion  tests.  The  number  of  hours 
a  trainee  might  spend  in  training,  or  the  amount 
of  information  he  or  she  is  exposed  to,  should 
have  no  effect  on  trainee  accountability.  The 
potential  impact  on  trainee  time  becomes  readily 
apparent. 


Use  of  a  CBT  System 


With  CRI  as  the  underlying  instructional 
philosophy  for  the  New  Hire  Program,  the  use  of  a 
Computer  Based  Training  (CBT)  system  for  admin¬ 
istration  of  the  program  was  considered.  After 
much  investigation.  United  management  determined 
that  a  CBT  system  was  essential  due  to  the 
complexity  of  the  training  task  and  the  necessity 
for  a  very  high  degree  of  control  over  our  target 
population-qualified  flight  crewmembers.  Errors 
in  record  keeping  of  performance  on  criterion 
tests  cannot  be  tolerated  in  our  environment.  All 
parties  (Company,  Federal  Aviation  Administration 
(FAA),  Instructors  and  Flight  Crew  members)  must 
be  assured  of  complete  satisfactory  performance  on 
all  criterion  tests. 

The  CBT  systems  available  were  evaluated  and 
Control  Data  Corporation's  PLATO  system  was 
selected  because  its  management  system  was  ideal ly 
set  up  to  conduct  our  application  for  newly  hired 
pilots.  PLATO'S  Computerized  Managed  Instruction 
(CMI)  system  administers  the  criterion  tests, 
prescribes  training  materials  and  monitors  trainee 
progress.  The  CMI  system  is  pre-programmed,  and 
as  a  result,  input  can  be  accomplished  largely  by 
clerical  personnel.  In  addition,  set  up  of  the 
course  and  all  other  necessary  input  can  be 
handled  by  development  personnel,  thus  eliminating 
the  need  for  computer  programmers. 


NEW  HIRE  COURSE  DESCRIPTION 


United's  New  Hire  Training  consists  of  28 
modules,  each  of  which  is  further  broken  down  into 
Objectives  or  Instructional  Units  (IUs).  Each  IU 
has  identified  with  it  one  or  more  Learning 
Resources  (LRs).  The  content  consists  of 
Orientation,  Advanced  Aeronautical  and  Job  Related 
subjects. 


New  Hire  Criterion  Testing 

The  criterion  testing  is  accomplished  on 
the  Module  level  but  LRs  are  prescribed  on  the  IU 
level.  For  example,  if  a  Module  contains  4  IUs,  a 
test  is  administered  for  each  IU,  but  mastery/non¬ 
mastery  scoring  takes  place  on  the  Module  level; 
that  is,  a  student  is  tested  on  a  Module  basis  and 
cannot  choose  an  individual  IU  test.  The  PLATO 
system  keeps  track  of  test  mastery  and  when  the 
Module  is  completed,  indicates  to  the  trainee 
which,  if  any,  of  the  IU  tests  have  been  mastered. 
For  the  unmastered  IUs,  PLATO  prescribes  LRs  which 
will  help  the  trainee  learn  the  content  in 


preparation  for  his/her  next  attempt  at  a  test. 
The  next  time  a  trainee  takes  a  test,  only  the 
unmastered  IU  tests  are  presented.  In  this 
example,  if  three  of  the  four  IU  tests  are 
mastered  on  the  first  attempt,  the  trainee  will 
be  tested  only  on  the  one  unmastered  IU  on  the 
next  attempt.  The  next  test  covers  the  same 
objective  but  consists  of  a  different  set  of 
questions.  If  a  trainee  fails  this  test,  he/she 
is  locked  out  of  further  testing,  and  is  either 
given  an  alternate  LR,  or  discusses  the  content 
with  a  subject  matter  expert.  Criterion  for  test 
mastery  varies  within  Modules,  but  the  range  is 
80%  to  90%. 

New  Hire  Learning  Resources 


Learning  Resources  for  the  Course  consist  of 
slide-tapes,  video-tapes,  emergency  equipment  and 
written  materials.  Most  of  these  were  developed 
specifically  for  this  course,  but  some  of  them 
were  chosen  from  existing  materials.  Individual 
study  carrels  in  a  Learning  Center  are  available 
for  the  indi vidual ized  instruction. 


New  Hire  Course  Options 


When  going  through  this  course,  the  trainee 
has  the  option  of  taking  the  test  first  or 
studying  the  LRs  first.  Since  the  trainee 
receives  the  course  objectives  prior  to  start-up, 
he/she  can  determine  his/her  course  of  action 
based  on  individual  knowledge  repertoire.  For 
those  tests  that  are  mastered  first,  the  trainee 
is  not  required  to  take  any  LRs,  but,  of  course, 
is  not  prohibited  from  using  them.  In  this 
manner,  training  is  tailored  to  individual  needs, 
and  training  time  is  not  wasted  in  areas  where 
proficiency  is  demonstrated. 


RESULTS  OF  NEW  HIRE  COURSE 


This  New  Hire  Course  was  conducted  from 
January,  1978  to  October,  1979.  (The  Course  has 
been  suspended  due  to  a  lack  of  need  for  new 
Second  Officers).  835  trainees  completed  the 
training  in  an  average  time  of  9.5  elapsed  days 
with  a  range  from  4  to  15  days.  The  conventional 
stand-up  course  that  was  replaced  by  this  CRI 
course  took  26  elapsed  days.  In  addition,  three 
instructors  who  were  assigned  for  subject  matter 
expertise  were  rarely  consulted  and  were  replaced 
by  personnel  for  coordination  at  signi  f  icantly 
lower  salaries. 
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Benefits  and  Advantages 


The  benefits  and  advantages  of  the  New  Hire 
Course  were  significant: 

*  Reduced  training  time  allowed  trainees  to 
move  into  a  transition  program  earlier, 
thereby  sending  well  qualified  pilots  into 
line  operation  much  sooner. 

*  Reduced  instructor  staff  eliminated  the  need 
to  replace  the  instructor’s  original 
positions.  All  instructors  returned  to  the 
jobs  they  had  before  becoming  members  of  the 
New  Hire  staff. 

*  Instruction  is  consistent.  All  trainees 
receive  (if  needed)  the  same  instruction  and 
all  tests  are  completed  to  mastery.  This 
eliminated  the  conventional,  subjective 
judgments  made  by  instructors.  Trainees 
preferred  the  new  course  since  they  were 
responsible  for  identical  material  and  were 
judged  consistently. 

*  PLATO  CM I  kept  all  trainee  records  current. 
A  trainee  knew  exactly  where  he/she  was  at 
all  times  and  could  plan  appropriately. 
Management  was  able  to  see  exactly  where 
trainees  were  at  a  given  moment  and  could 
use  this  data  to  provide  even  more 
individual  help  if  needed. 

Trainees  were  more  relaxed  and  enjoyed  the 
individual  interaction  with  other  trainees 
that  occurs  spontaneously  with  an 
individualized  program. 

*  Changing  of  poorly  worded  or  unclear 
questions,  misspellings,  and  all  other 
necessary  editing  corrections  was 
accomplished  on  the  spot  in  PLATO.  This 
eliminated  the  problem  of  making  pencil 
changes  to  what  would  have  amounted  to 
multiple  hard  copies. 

*  Obviously,  money  was  saved.  Exactly  h 
much  was  not  specifically  determined  due  t>. 
complex  accounting.  Trainees  became  line 
crew  members  more  quickly  than  anticipated, 
and  as  a  result,  more  flights  were  flown, 
more  passengers  boarded,  etc.  However,  it 
was  conservatively  estimated  that,  after 
paying  for  the  use  of  PLATO,  well  over 
$100,000  a  year  would  be  saved  just  due  to 
Instructor  salaries  and  less  trainee  time 
spent  in  training  Instead  of  being 
productive  line  pilots. 


Disadvantages 


Two  major  disadvantages  existed  with  the 
New  Hire  Course  that  are  worth  noting: 

*  Due  to  a  managerial  decision  concerning 
security  of  the  criterion  tests,  trainees 
received  no  feedback  to  the  questions. 
They  did  not  know  which  questions  were 
answered  correctly  or  incorrectly.  This 
placed  them  in  more  of  a  straight  testing 
mode  rather  than  in  a  combination 
testing/learning  process  which  we  consider 
more  desirable.  Many  students  felt  this 
lack  of  feedback  was  inappropriate  and,  at 
times,  caused  some  minor  frustrations. 

*  We  did  not  use  Computer  Assisted 

Instruction  (CAI)  in  this  Course.  This  was 
due  to  the  lack  of  programming  personnel 
available  whi  'old  work  with  the  PLATO 
language.  Thi  jynamic  feature,  CAI, 

of  a  CBT  system  , d  have  provided  trainee 
-  PLATO  interaction  which  would  have  been 
more  meaningful  to  the  trainee  in  the 
learning  process. 


FAA  Acceptance 


The  advantages  of  the  New  Hire  Course, 
however,  strongly  outweighed  the  disadvantages, 
and  the  course  was  widely  accepted  throughout  the 
company  and  with  the  FAA.  The  FAA  regulates  all 
of  our  courses  and  approves  general  content  and 
number  of  hours  required  for  a  given  course. 
Since  we  could  not  provide  a  specific  number  of 
hours  for  New  Hire  Course  completion  due  to  the 
CRI  and  individualized  concept,  FAA  approval  of 
the  design  was  a  major  change  in  their 
regulations.  This  was  considered  to  be  a 
significant  breakthrough  in  the  agency's  somewhat 
traditional  approach  to  training,  and  it  paved 
the  way  for  us  to  continue  in  our  pursuit  of 
applying  the  CRI/CMI  approach  to  other  courses  at 
the  Training  Center. 


INTRODUCTION  TO  IFO 


The  next  course  to  be  redeveloped  was  the 
Initial  First  Officer  (IFO)  Course  which  is 
designed  to  upgrade  Second  Officers  into  First 
Officers  positions.  Course  content  consists  of  a 
review  of  Meteorology,  Radar,  and  United's  Flight 
Operations  Manual  (FOM),  and  contains  15  modules 
broken  down  into  a  nunfcer  of  IUs.  It  is  similar 
to  the  New  Hire  Course  in  that  the  principles  of 
CRI  were  used  for  design  and  development,  and 
PLATO  CMI  is  used  for  test  administration,  LR 
prescription  and  record  keeping.  There  are 
significant  differences,  however,  between  the  two 
courses  in  the  areas  of  criterion,  learning 
resources  and  feedback. 
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IFO  COURSE  DESCRIPTION 


Here's  how  the  IFO  Course  works: 

As  in  New  Hire,  the  trainee  has  the  option 
of  taking  any,  or  all,  of  the  LRs  first,  or,  if 
after  reading  the  objectives,  of  taking  the  test 
first.  In  the  IFO  Course,  however,  all  of  the  LRs 
are  CAI  lessons  instead  of  the  multi-media  LRs  of 
the  New  Hire  Course.  The  use  of  CAI  allowed  the 
Program  Development  staff  to  utilize  the  inter¬ 
action  feature  of  PLATO  to  improve  instruction. 
It's  also  much  more  efficient  for  the  trainee  to 
have  everything  right  in  front  of  him  in  one 
carrel  using  one  piece  of  equipment.  The  content 
of  the  Course  was  such  that  PLATO  graphics  and  a 
supplemental  book  of  drawings  and  pictures  could 
easily  replace  the  more  expensive  use  of 
slide-tapes  and  video-tapes. 


IFO  Criterion  Tests 


The  criterion  tests  in  the  IFO  Course  have 
two  sets  of  questions,  an  "A"  set  and  a  "B"  set. 
Typically,  the  trainee  will  be  presented  the  "A" 
question  first,  let's  say  "1A.  If  he/she  gets  it 
correct,  it  will  be  so  noted,  and  he/she  will 
continue  on  to  "2A."  If  he/she  gets  it  wrong,  it 
will  be  so  noted,  and  the  trainee  will  receive 
Feedback  (FB)  explaining  content  related  to  the 
missed  question.  After  the  FB,  the  next  "A" 
question  is  presented.  This  procedure  continues 
until  the  end  of  the  test.  After  all  of  the  "A“ 
questions  are  presented,  the  trainee  receives  the 
"B"  questions  that  directly  relate  to  the  missed 
question(s).  If  any  ”B"  question  is  missed,  the 
test  is  unmastered  and  the  trainee  is 
automatically  locked  out  of  further  testing  in 
that  Module,  and  must  take  the  appropriate  LR(s) 
before  being  retested.  This  cycle  continues, 
then,  until  mastery  is  achieved;  that  is,  until 
no  "A"  or  "B"  questions  are  missed. 


RESULTS  OF  IFO  COURSE 


The  CRI/IFO  Course  was  conducted  from 
January,  1980  to  May,  1980.  (Classes  have  been 
suspended  for  a  short  time  due  to  lack  of  need  for 
new  First  Officers).  38  trainees  completed  the 
training  in  an  average  time  of  4.5  elapsed  days 
with  the  range  from  3  to  7  days.  The 
conventional,  stand-up  course  that  was  replaced  by 
this  CRI  course  took  10  elapsed  days.  In 
addition,  two  instructor  positions  were 
eliminated;  coordination  of  the  course  is  provided 
by  the  same  personnel  who  administer  the  New  Hire 
Course. 


COMPARISON  OF  NEW  HIRE  AND  IFO 


Because  of  the  CRI  design  and  the 
individualization  of  the  IFO  Course,  the  benefits 
and  advantages  attained  were  similar  to  those 
accomplished  with  the  New  Hire  Course.  The 
disadvantages  cited  in  the  discussion  of  New 
Hire,  however,  were  eliminated  in  the  IFO 
Course. 


Difference  in  Feedback 


In  New  Hire,  no  feedback  to  the  questions 
was  provided;  in  IFO,  not  only  were  the  trainees 
given  correct /incorrect  answer  information,  they 
were  also  given  immediate  instruction  relating  to 
the  subject  matter  of  the  question  missed.  As  a 
result,  trainees  knew  immediately  how  well  they 
were  doing  and  learned  why  they  answered  a 
question  incorrecty.  Their  learning  was  then 
reinforced  when  they  had  to  answer  a  similar 
question  to  the  one  missed  within  a  reasonably 
short  period  of  time.  Thus,  trainees  in  the  IFO 
Course  were  placed  in  a  t est i ng/ 1  ea rn i ng 
environment  as  opposed  to  a  strictly  testing 
situation  (as  in  New  Hire),  and  the  Course  was 
more  meaningful  to  them.  They  were  also  more  at 
ease  because  they  knew  how  they  were  progressing 
with  each  question  and  didn't  feel  the  pressure 
of  not  knowing  how  they  were  doing  until  the  test 
was  completed,  as  was  the  case  with  New  Hire. 


Differences  in  Learning  Resources 


Another  major  difference  between  the  two 
courses  was  the  Learning  Resources.  While  some 
of  the  slide-tapes  were  appropriate  for  the 
subject  matter  in  New  Hire,  many  of  them  were 
unnecessary,  but  were  done  that  way  simply 
because  that's  the  way  we  had  done  them  for  many 
years.  Slide-tapes  were  accepted  and  were 
considered  the  appropriate  medium  for  most 
learning  situations.  While  designing  and  devel¬ 
oping  the  IFO  Course,  we  determined  that  PLATO 
graphics  available  through  CAI  were  as 
appropriate,  if  not  more  so,  for  most  of  the 
content  than  were  slide-tapes.  Trainees  felt 
that  the  interaction  with  PLATO  was  extremely 
valuable  and  meaningful,  and  they  were  less  bored 
as  a  result.  Also,  editing  was  handled  quickly 
and  efficiently,  trainees  did  not  have  to  cope 
with  potential  hardware  problems  and  the  lessons 
were  easily  accessible  and  readily  available. 


231 


Difference  in  Criteria 


ABOUT  THE  AUTHOR 


Finally,  the  criterion  established  for  each 
course  was  different.  In  New  Hire,  the  criterion 
for  module  mastery  ranged  from  80*  to  90%  while 
in  IFO,  the  criterion  was  effectively  100%.  In 
New  Hire,  when  a  trainee  reached  the  necessary 
criterion  while  taking  a  test,  the  test  was 
stopped  and  the  trainee  did  not  see  the  remaining 
questions.  In  IFO,  the  trainee  saw  all  of  the 
questions  and  had  to  answer  correctly  all  of  the 
questions  or  alternates  in  order  to  master  a 
module.  In  this  case,  we  were  assured  that  the 
trainees  were  exposed  to  all  of  the  needed 
information,  and  they  attained  mastery  only  after 
responding  correctly  to  100%  of  the  questions. 


CONCLUSIONS 


Both  courses,  despite  the  differences, 
accomplished  their  objectives.  Trainees  who 
completed  training  continued  on  to  the  next 
phases  with  the  knowledge  necessary  to 
successfully  become  new  Second  and  First 
Officers.  Further,  both  courses  were  objective 
appraisals  of  trainee  knowledge.  Prior  to  these 
courses,  mere  completion  of  a  course  was  adequate 
criterion  for  success.  With  the  new  courses, 
trainees,  under  more  pressure,  were  somewhat 
apprehensive  at  first,  but  most  of  them 
appreciated  the  logic  behind  the  rationale  for 
objectivity.  As  with  most  professionals,  when 
given  an  opportunity  to  "prove"  themselves,  our 
trainees  overcame  any  misgivings  they  had,  and 
completed  the  courses  with  minimal  difficulty. 


FUTURE  APPLICATIONS 


Due  largely  to  the  success  of  these  two 
courses,  United  has  committed  itself  to  the  use 
of  PLATO  in  the  future.  Two  more  applications 
are  underway  -  the  Initial  Captain  Course  and  the 
DC-10  Flight  Guidance  System  are  being  developed 
using  CRI/CMI/CAI. 

In  addition,  PLATO's  largest  contribution  will  be 
with  our  new  airplane  program  -  the  Boeing  767. 
Not  only  will  PLATO  be  used  in  a  similar  manner 
as  described  in  this  paper,  but  it  will  also  be 
used  in  the  simulator  as  an  interface  with  the 
simulator  computer,  and  will  store  all  trainee 
records  throughout  the  entire  training  program. 
The  767  program  will  be  on-line  in  early  1982  and 
it  s  anticipated  that  the  program  design, 
development,  and  implementation,  using  PLATO  as 
an  integral  part  will  be  the  most  modern  airplane 
training  program  in  existence.  The  use  of  the 
CRI/CMI/CAI  concept  will  account  for  the  entire 
program  and  should  prove  to  be  the  most  meaning¬ 
ful  ,  successful  program  ever  developed  in  the 
airline  industry. 


David  A.  Kolakowski  is  a  Staff  Flight  Training 
Program  Development  Specialist  at  United  Airlines 
Flight  Training  Center,  Denver,  Colorado,  and  is 
responsible  for  the  design,  development, 
validation  and  implementation  of  training 
programs  for  Flight  Crews.  Mr.  Kolakowski  holds 
a  Master's  degree  in  education  and  a 
Turbojet -rated  Flight  Engineers  Certificate. 
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NEW  DIRECTIONS  IN  TRAINING 
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Marine  Corps  Communication-Electronics  School 
Marine  Corps  Air  Ground  Combat  Center 
Twentynine  Palms,  California  92278 

ABSTRACT 

The  Mastery  Learning  Project  at  the  Marine  Corps  Communication-Electronics  School  in 
Twentynine  Palms,  California  is  the  Marine  Corps'  test  bed  for  the  development  of  a  self-paced, 
mastery- learning  strategy.  This  strategy  is  viewed  as  the  way  to  improve  personnel  and  train¬ 
ing  readiness  by  providing  complex  technical  instruction  in  less  time,  with  less  academic 
attrition,  and  with  no  degradation  in  quality.  The  key  to  success  lies  in  the  emphasis  placed 
on  offering  closer,  more  personalized  instructor  attention  to  those  students  who  require  it. 

A  computer-based  management  information  system  provides  essential  automated  support  for 
instructional  personnel  as  well  as  for  all  the  administrative  and  managerial  functions  within 
the  school,  thereby  permitting  substantial  reductions  in  training  overhead  costs.  Experience 
gained  thus  far  has  shown  that  students  are  achieving  mastery  of  all  learning  objectives  in 
70%  of  the  conventional  training  time,  with  academic  attrition  approaching  zero. 


Since  World  War  II,  the  Iwo  Jima  Memorial, 
located  just  outside  of  Washington,  O.C.,  has 
been  a  symbol  of  Marine  Corps  valor.  But  more 
than  that,  it  has  stood  as  a  reminder  of  our 
country's  readiness  to  meet  and  conquer  any 
threat  to  national  security  and  to  those  values 
and  principles  we  cherish.  In  recent  months, 
however,  our  readiness  has  become  a  matter  of 
grave  concern--can  we  still  respond  militarily 
in  time  of  crisis?  This  leads  to  the  question: 
What  is  readiness? 

Very  simply  stated,  readiness  is  the 
synthesization  of  equipment,  people,  and 
training.  The  most  sophisticated  equipment  in 
the  marketplace,  along  with  the  necessary  people 
to  operate  and  maintain  it,  will  not  produce 
readiness  without  an  effective  training  system. 
In  a  certain  sense,  then,  we  can  say  that 
readiness  equates  to  training. 

As  we  enter  the  1980s,  military  training  is 
becoming  much  more  challenging.  Specialized 
skills  training,  in  particular,  demands  more 
imaginative  responses  to  constantly  changing 
requirements.  The  test  bed  for  a  long-range 
Marine  Corps  program  to  examine  new  directions 
in  training  which  will  respond  to  this  challenge 
is  the  Computer  Based  Education  (CBE)  Project  at 
the  Marine  Corps  Communication-Electronics 
School  in  Twentynine  Palms,  California. 

The  conventional,  lockstep,  class-oriented 
approach  to  instruction  has  traditionally  been 
the  standard  way  of  conducting  training  in  the 
Marine  Corps.  It  has  served  its  purpose  well, 
but  it  is  rapidly  becoming  ineffective  in  the 
face  of  demands  being  placed  upon  it  by  three 
major  factors:  technology,  manpower,  and  money. 
Let  us  examine  each  of  these. 

Technology  is  changing  the  world  we  live  in 
moment  by  moment.  However,  the  change  is 
insidious.  It  is  happening  so  fast  that  it 
blurs  into  a  gradual  continuum  and  is  upon  us 
before  we  know  it.  As  with  our  children,  growth 
occurs  a  little  each  day  and  we  never  notice 
it...  until  one  evening  when  we  sit  down  to  the 
dinner  table,  we  find  they  are  no  longer  there. 
We  are  astounded  that  they  have  become  adults-- 
overnight!  How  many  people  still  use  a 
wristwatch  with  a  stem  winding  mechanism?  The 
numbers  are  rapidly  diminishing.  And  it  seems 
only  yesterday  that  the  first  electronic 
calculator  fairly  dominated  the  entire  surface 


of  a  double  pedestal  desk.  Now  you  can  fit  one 
in  your  wallet.  When  did  this  all  happen?  No 
question  about  it,  the  technology  explosion  has 
vastly  expanded  our  operational  capabilities, 
but  it  has  created  a  dependence  at  the  expense 
of  self-reliance.  For  some,  the  deprivation  of 
the  automatic  garage  door  opener  or  the  remote 
TV  tuner  would  be  a  shattering  experience.  The 
pricetag  for  technology  does  not  end  there.  The 
more  complex  systems--such  as  those  used  for 
military  command  and  control  systems--cost 
dearly  and  require  more  highly  trained 
personnel.  In  short,  technology  is  a 
double-edged  sword.  But  then,  you  don't  get 
nuthin'  fer  nuthin' ! 

It  has  been  humorously  commented  that  if 
the  safety  pin  were  invented  today,  it  would 
have  two  transistors,  a  regulator,  an  off-on 
switch,  and  would  require  a  service  check  every 
six  months.  This  is  not  meant  to  poke  fun  at 
technology,  but  merely  to  underscore  the  fact 
that  everything  is  becoming  more  complex 
nowadays.  How  does  technology  impact  on 
national  defense?  The  U.S.  has  made  the 
decision  not  to  compete  with  the  Soviet  Union  in 
the  production  of  conventional  arms. 

Accordingly,  it  can  only  maintain  its 
operational  edge--sustain  its  "essential 
equivalency "--through  the  use  of  technology. 

This  means  that  greater  and  greater  numbers  of 
multi-purpose,  technologically  sophisticated 
weapons  systems  are  entering  the  military 
equipment  inventory.  Just  as  the  cannon  was 
developed  to  breach  forts  and  later  on  the  tank 
was  invented  to  breach  static  defenses,  now  a 
whole  new  generation  of  military  capabilities  is 
being  created. . .which  in  turn  will  breed  counter 
capabilities. . .and  so  on.  These  new  systems  are 
not  merely  product  improvements  on  existing 
systems  but  are  in  many  cases  totally  new, 
add-on  systems  which  further  expand  the  number 
of  equipment  end  items  to  operate  and  maintain. 
The  greater  sophistication  of  these  systems 
makes  their  operation  and  maintenance  tasks  much 
more  complicated  and  difficult  to  master  and 
leads  to  a  greater  diversification  of  skills. 

The  generalist  is  fast  giving  way  to  the 
specialist.  In  short,  greater  numbers  of  more 
complicated  systems  require  greater  numbers  of 
more  highly  skilled  personnel  to  support 
them. ..essentially  fewer  and  fewer  "oick  and 
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shovel  jobs". 

One  very  grave  implication  of  the  rapid 
advances  being  made  by  technology  is  that  the 
individual  is  given  greater  control  over  his 
environment.  Where  once  the  archer  could 
control  anything  within  range  of  his  arrow,  and 
then  the  pilot  had  control  over  anything  within 
range  of  his  aircraft,  now  the  missileman  can 
control  those  targets  within  range  of  his 
Inter-Continental  Ballistic  Missile  (ICBM). 
Although  this  is  an  Impressive  operational 
advancement,  it  carries  with  it  the  serious 
burden  of  increased  consequences  due  to  human 
error.  What  does  this  mean?  Simply  that  we 
cannot  afford  to  have  poorly  or  partially 
trained  individuals— each  individual  must  be 
100%  proficient  in  his  specialty.  In  summary, 
technology  is  generating  greater  and  more 
complex  training  requirements. 

Manpower  is  the  second  factor  which  is 
driving  the  requirement  to  investigate  new 
directions  in  training.  There  are  three  elements 
to  be  considered... Quantity...  Quality. . .and 
Atti tude. 

We  have  been  operating  without  the  Draft 
since  1973,  and  the  validity  of  the  All 
Volunteer  Force  is  being  seriously  questioned. 
Why?  Studies  have  shown  that  in  order  to 
maintain  the  2-m:11ion  plus  standing  force, 
approximately  340,000  recruits  must  be  enlisted 
each  year.  This  equates  to  about  1300  recruits 
per  working  day... a  lot  of  volunteers! 
Paradoxically,  the  manpower  pool  from  which 
these  volunteers  must  be  drawn  is  rapidly 
decllm'ng—there  are  simply  fewer  17-18  year 
olds  than  in  the  past  few  decades.  It  has  been 
estimated  that  by  the  mid-1980s,  one  out  of 
every  three  qualified  18-year-old  must  be 
enlisted  in  order  to  support  the  All  Volunteer 
Force.  To  compound  the  manpower  quantity 
problem  is  the  fact  that  retention  of  mid-career 
military  professionals  is  becoming  more  and  more 
difficult. ..especially  among  the  so  called  "hard 
skills"  such  as  electronics  maintenance.  The 
lure  of  higher-paying,  less-demanding  civilian 
jobs  is  hard  to  combat. 

The  second  element  of  the  manpower  issue  is 
quality.  Bluntly,  fewer  and  fewer  new  recruits 
have  the  necessary  math  or  verbal  skills  to 
successfully  complete  demanding  technical 
training.  Since  1957,  the  scores  registered  by 
high  school  students  on  the  Scholastic  Aptitude 
Test  (SAT)  have  been  plummeting.  This  lack  of 
qualifications  further  narrows  the  manpower 
pool.  It  is  not  that  the  young  people  of  today 
are  stupid--they  are  just  poorly  educated.  You 
can  read  about  it  every  day  in  your  newspapers 
and  periodicals.  One  Ohio  State  University 
study  revealed  that  approximately  25%  of  a 
recent  freshman  class  had  to  take  remedial 
reading—nearly  one- third  had  to  take  make-up 
math!  Another  study  indicates  that  almost  13% 
of  all  17-year  olds  are  functionally  illiterate. 
This  leads  to  the  irrefutable  conclusion  that 
the  high  school  diploma  is  not  what  it  used  to 
be.  And  again...  among  the  qualified,  many  are 
lured  away  by  more  profitable  careers. 

The  final  aspect  of  manpower  which  features 
Into  the  set  of  considerations  which  .represent 
the  training  challenge  is  attitude.  The  Marine 
Corps  seeks  individuals  who  fit  the  adage:  When 
the  going  gets  tough,  the  tough  get  going! 
However,  we  are  seeing  an  ever-increasing 


incidence  of  an  attitudinal  problem  among  our 
young  enlistees  which  the  Commandant  of  the 
Marine  Corps  has  very  aptly  dubbed  the  "failure 
stick  syndrome."  This  is  reflective  of  the 
growing  need  within  our  society  for  instant 
gratification  by  the  easiest  route— a 
manifestation  of  the  TV-oriented,  fast- food 
mentality.  Just  like  the  "hired  gun”  in  the  old 
western  movies,  who  carved  a  notch  in  his  gun 
each  time  he  killed  someone,  many  young  people 
today  seem  to  take  some  sort  of  perverse  pride 
in  the  number  of  notches  they  can  carve  in  their 
"failure  sticks’1— to  be  success  at  failure,  as 
it  were.  The  obvious  implication  this  has  in 
our  discussion  is  that  technical  training  and 
learning  do  not  take  place  effortlessly,  a  high 
degree  of  mental  toughness  is  required.  Conse¬ 
quently,  we  find  more  and  more  cases  of  students 
who  simply  quit  because  the  learning  tasks 
appear  to  be  too  difficult  and  demanding.  Mind 
you,  many  of  these  have  the  necessary  cognitive 
abilities  to  be  successful;  their  failures  can 
only  be  attributed  to  poor  attitude  and  lack  of 
motivation. 

The  third  major  factor  which  contributes  to 
the  situation  facing  the  military  training 
establishment  is  money.  In  the  first  place,  a 
shift  has  taken  place  since  the  early  Vietnam 
days  in  terms  of  the  budget  outlays  provided  to 
the  Department  of  Defense  and  the  amounts  set 
aside  for  various  human  resource  programs. 
Defense  spending  has  fallen  from  42%  to  23%  of 
the  Federal  Budget  while  human  resource 
allocations  have  climbed  from  29%  to  53%.  The 
current  Iranian  crisis  has  exposed  potential 
weaknesses  within  the  Department  of  Defense  and 
resulted  in  calls  to  raise  defense  spending. 
Although  the  military  now  enjoys  a  slightly 
larger  slice  of  the  Gross  National  Product 
(GNP),  pressure  on  the  Congress  to  balance  the 
1981  budget  is  constant.  As  a  result,  any 
improvements  may  be  short  lived.  As  though 
proportionate  quantitative  cuts  in  the  DOD 
budget  were  not  enough,  double  digit  inflation 
compounds  the  matter  even  further...  as  it  does 
in  every  walk  of  life.  Thus,  the  defense  budget 
is  hit  from  two  sides.. -and  within  the  defense 
budget,  the  high  acquisition  costs  of  complex 
weapons  systems  normally  cause  a  concomitant 
reduction  in  dollars  available  for  training. 

A  major  military  training  center,  such  as 
the  Marine  Corps  Communication-Electronics 
School,  is  faced  with  the  challenge  of  providing 
expanded  and  more  complex  training  to  fewer, 
less  qualified,  and  less  motivated  students  for 
less  money.  M-C-C-E-S  or  C-and-E  School,  as  it 
is  called  in  the  vernacular,  is  an  expansive 
facility  with  the  mission  of  providing  training 
.. .electronics  fundamentals .. .operational 
conmuni  cati  ons . . .air  control/anti-air  warfare 
operati ons . . . and  conmuni cati on-el ectroni cs 
maintenance.  Very  broadly,  C  S  E  School  is 
responsible  for  training  approximately  85%  of 
the  personnel  who  operate  or  maintain  the  Marine 
Corps’  comnand  and  control  conmunicatlons 
systems.  It  conducts  45  courses  of  Instruction 
leading  to  37  Military  Occupational  Specialities 
(M0Ss)...with  several  more  courses  under 
development.  M-C-C-E-S  has  a  yearly  input  of 
more  than  6000  plus  students  with  a  peak  load 
exceeding  2100.  The  average  daily  student  load 
is  approximately  1700  Marines.  C  &  E  is 
authorized  a  permanent  staff  of  51  officers. 
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to  becoming  better  educated  in  the  area  of 
computer  based  training,  to  conducting  research, 
and  to  developing  a  solid  5-year  plan  for  the 
acquisition  and  implementation  of  our  future 
system.  We  studied  all  the  significant 
literature  we  could  get  our  hands  on,  we 
attended  seminars  and  conferences,  and  we 
visited  most  of  the  major  civilian  and  military 
institutions  involved  in  similar  programs.  We 
also  received  solicited  and  unsolicited  visits 
by  a  number  of  highly  respected  experts  in  the 
fields  of  instructional  technology  and 
educational  psychology.  All  very  willingly  lent 
their  professional  opinions  and  confirmed  that 
we  were  on  the  right  track. 

One  resounding  note  which  rang  loud  and 
clear  in  all  this  effort  was  that  any  such 
system  as  the  one  we  were  contemplating  had  to 
be  based  on  a  marriage  of  CA1  and  CMI-with  the 
emphasis  on  CMI.  To  avoid  possible  confusion, 
which  vie  perceived  could  arise  due  to  the 
varying  definitions  of  the  terms  CAI  and  CMI 
which  abound,  we  adopted  the  umbrella  term 
Computer  Based  Education  (CBE)  for  future 
reference  to  our  project.  Our  5-year  plan 
laid  out  our  objectives  with  milestones— 
as  we  saw  them  then-an d  set  forth  the 
essential  elements  of  our  CBE  system.  These 
consisted  of  six  major  subsystems,  each 
containing  various  application  programs.  The 
Training  subsystem,  of  course,  reflects  the 
basic  requirements  for  the  CAI/  CMI  system  set 
forth  in  our  systems  specifications;  Staff 
Management  addresses  the  accountability  and 
allocation  of  our  human  resources;  Resource 
Allocation  does  the  same  thing  for  our  material 
resources;  Instructor  Evaluation  provides  the 
quality  control  over  our  instructors— still  the 
key  element  in  our  instructional  strategy; 

Student  Management  automates  the  myriad 
administrative  tasks  which  must  occur  from  the 
time  we  learn  of  a  student's  assignment  to  the 
school  to  the  time  he  leaves;  and  Courseware 
Development  is  designed  to  facilitate  the 
monumental  task  of  updating  course  material. 

These  six  subsystems  embrace  all  of  the 
functions  which  we  define  as  the  "educational 
process"  and  represent  the  framework  for  our  CBE 
system. 

There  is  nothing  new  about  these 
functions—they  all  are  being  performed 
currently.  They  are  all  crucial  to  our  mission. 
However,  our  performance  of  these  functions  is 
fast  becoming  ineffective  in  an  educational 
environment  built  on  the  traditional,  lockstep 
method  of  instruction.  Why?  Because  the  entire 
approach  to  training  is  no  longer  responsive  to 
the  new  demands  being  placed  upon  it  by  those 
external  conditions  over  which  we  have  no  control. 

bet's  take  a  look  at  just  why  lockstep 
instruction  no  longer  answers  the  mail.  Very 
simply:  it  fails  to  train  fully  all  students 
and  it  is  fraught  with  high  academic  attrition. 
The  major  reason  for  this  is  that  it  is  based  on 
a  Fixed  Time/Variable  Mastery  strategy.  A 
course  is  conducted  over  a  set  time  period,  and 
those  students  who  do  graduate  attain  anywhere 
between  70%  and  1 00%  mastery  of  the  learning 
objectives.  Instruction  is  of  necessity  geared 
to  the  average  student.  This  means  that  it  is 
too  slow  for  approximately  the  top  one-third  of 
the  class  and  too  fast  for  the  bottom  one-third. 
The  highly  qualified  student  is  capable  of 


mastering  all  of  the  learning  objectives  in  much 
less  time  than  it  takes  the  course  to  run  to 
completion,  whereas  the  slow  student  becomes 
hopelessly  lost  before  he  ever  gets  started. 

The  good  students  get  bored,  many  never 
realizing  their  full  potential,  some  actually 
becoming  "academic"  attritions  themselves.  And 
the  most  serious  consequence  is  that  we  are 
sending  men  into  the  field  who  may  be  as  low  as 
70%  trained.  1  doubt  if  anyone  of  us  would  like 
to  be  treated  by  a  doctor  who  was  70%  qualified 
...or  fly  with  a  pilot  who  was  great  on 
take-offs  but  so-so  on  landings!  What's  the 
answer? 

We  believe  the  solution  has  its  foundation 
in  Mastery  Learning  theory.  Pioneered  by  Dr. 
Benjamin  Bloom  of  the  University  of  Chicago  and 
further  promoted  by  a  number  of  his  disciples, 
Mastery  Learning  holds  that  "all  can  learn." 

The  implication  is  that  by  adapting  our 
instructional  approach  to  the  learning 
capabilities  of  the  student,  we  can  bring 
all --or  nearly  all — up  to  the  level  of  mastery 
now  normally  attained  by  the  top  10%.  We  are 
confident  that  this  can  be  achieved.  But  how 
will  it  work? 

The  educational  strategy  which  supports 
Mastery  Learning  is  virtually  a  reversal  of  that 
which  drives  lockstep  instruction.  Whereas 
lockstep  strategy  is  based  on  Fixed  Time/ 
Variable  Mastery .. .Mastery  Learning 
strategy  is  based  on  Fixed  Mastery/Variable  Time 
(FM/VT).  In  the  FM/VT  approach,  mastery  of  all 
learning  objectives  is  a  fixed  requirement,  the 
time  to  master  varying  according  to  the  unique 
learning  capabilities  of  each  student.  When 
full  mastery  is  attained—measured  along  the  way 
by  successful  completion  of  a  series  of 
formative  exams  and  at  the  end  by  passing  a 
final,  summative  exam-the  student  moves  on.  The 
key  to  control  under  the  FM/VT  strategy— and 
that  ingredient  which  prevents  Marines  from 
becoming  "career  students"— lies  in  the 
interactive  testing  scheme.  This  disallows  a 
student  to  exit  a  question  on  an  exam  with  the 
wrong  answer.  The  student  is  given  immediate 
feedback,  and  problem  areas  are  identified, 
signalling  the  need  for  instructor  intervention. 
As  opposed  to  the  image  conjured  up  by  the  term 
"self-paced”  instruction,  which  picture*  a 
student  isolated  in  an  antiseptic  room,  alone 
with  his  computer--FM/VT  emphasizes  the  need  for 
the  ever-present  instructor,  and  allows  him... to 
employ  the  one-to-one  Socratic  method  of 
instruction  with  those  students  who  need  it. 
Since  this  age-old,  ideal  mode  of  instruction  is 
impossible  to  employ  on  a  mass  scale,  FM/VT 
permits  us  to  implement  it  "by  exception." 

On  September  1st,  1979,  M-C-C-E-S  launched 
into  Phase  II  of  the  project--the  Implementation 
Phase.  With  announcement  of  the  system's  vendor 
imminent  and  delivery  of  the  first  increment  of 
the  system  hardware  approximately  a  year  away, 
we  knew  we  had  much  work  to  do  to  prepare  for 
its  arrival.  The  project  team  was  augmented  by 
additional  personnel  and  restructured  into  four 
functional  "tracks"  under  the  direct  management 
of  the  Commanding  Officer. 

Track  1  was  designated  the  Hardware  Track, 
responsible  for  making  all  the  preparations 
necessary  to  acconmodate  the  equinment  when  it 
arrived.  It  was  also  tasked  to  handle  all  the 
supply  and  maintc-a  <ce  requirements  for  the 
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463  enlisted  personnel,  and  48  civilian  employees 
to  support  this  immense  training  responsibility. 
However,  normal  staffing  runs  at  a  somewhat 
reduced  level  based  on  the  availability  of 
various  ranks  and  MOSs  within  the  Marine  Corps. 

The  inpact  of  Technology,  Manpower,  and 
Money  on  M-C-C-E-S  can  be  driven  home  by  the 
example  of  a  radar  repairman.  Fifteen  years  ago 
he  could  fix  all  of  the  radars  in  the  Marine 
Corps  inventory  with  one  toolbox. . .now,  radar 
repair  requires  a  whole  array  of  skills  and 
equipment.  Very  simply,  the  increased 
complexity  of  technology. . .and  the  greater 
diversity  of  more  sophisticated  systems  are 
vastly  expanding  the  range  and  level  of 
technical  skills  required  by  the  Marine 
Corps... with  a  commensurate  increase  in  the 
number  of  personnel  who  must  be  trained. 

One  final  aspect  of  technology  which  has 
manpower  and  money  implications  and  which 
impacts  on  the  training  mission  at  M-C-C-E-S  is 
the  increasing  speed  of  technological  change. 
Scientific  advances  are  hurtling  forward  in 
geometric  leaps  and  bounds... to  the  extent  that 
the  effectiveness  span  of  an  individual's 
knowledge,  skill,  and  training  is  greatly 
reduced;  simply  stated,  technical  training  has  a 
built-in  decay  factor.  This  requires  more 
frequent  updating  of  training  and  higher  costs. 
For  example,  during  the  decades  between  1930  and 
1950,  tube  technology  was  the  existing  state  of 
the  art.  For  the  most  part,  a  technician  who 
entered  the  service  in  1930  could  still  operate 
effectively  20  years  later  without  much,  if  any, 
formal  retraining.  Then  in  the  late  1950s  we 
advanced  to  mini  tubes.  By  the  early  1960s  we 
had  moved  into  the  world  of  transistors  and 
medium  discrete  components ...  the  first  attempts 
at  integrated  circuits.  1965  found  integrated 
circuits  in  full  bloom. ..and  saw  the 
introduction  of  layered  hybrids. . .analog 
techniques  being  replaced  by  digital  concepts. 

By  1970  we  had  reduced  mul ti- integrated  circuits 
to  a  single  chip...  and  were  compressing  layers 
of  separate  circuitry  into  multilayer  boards  by 
the  mid- 70s. ..and  this  level  of  technology  is 
now  obsolete!  Thus,  as  technological 
advancements  rapidly  accelerate  on  the  one  hand, 
and  the  pool  of  "first  choice"  Marines  steadily 
declines  on  the  other  hand,  C  &  E  School  must 
teach  a  more  complicated  array  of  subjects  to 
lesser  quality  students.  This  reduction  in 
quality  leads  to  higher  attrition  and  longer 
training  times,  increased  in  part  by  the 
requirement  to  provide  remedial  training.  The 
increased  pressure  to  learn  greater  amounts  of 
more  difficult  course  material  also  takes  its 
toll  in  attrition  among  the  qualified  but  poorly 
motivated  Marines.  The  higher  pay  of  staff 
personnel  and  students  leads  to  higher  training 
overhead  costs,  especially  in  the  manpower- 
intensive  conventional  lockstep  method 
of  instruction.  Inflation,  too,  is  increasing 
operating  costs  at  C  &  E  School  without  an 
increase  in  productivity. ..  which,  in  turn, 
breeds  further  inflation.  Therefore,  the 
expanded  training  burden  brought  about  by 
technological  progress,  juxtaposed  with  the 
depressed  manpower  situation,  creates  higher 
training  costs... all  at  a  time  when  money  is 
becoming  more  scarce.  Our  only  recourse  is  to 
train  quicker,  better,  cheaper. 

From  the  point  of  view  of  the  trainer,  this 


poses  what  appears  to  be  an  insurmountable 
problem.  We  have  no  control  over  the  march  of 
technology...  we  have  no  control  over  the 
quantity,  quality,  and  attitude  of  the  manpower 
we  receive. . .and  we  have  no  control  over  the 
money  we  are  allocated.  What,  then,  can  we 
possibly  do?  On  the  surface  it  appears  that  we 
are  faced  with  an  insoluable  situation  imposed 
on  the  training  establishment  by  the  factors  of 
Technology,  Manpower,  and  Money.  This  situation 
did  not  develop  overnight--as  long  as  ten  years 
ago,  the  Marine  Corps  perceived  that  something 
had  to  be  done  to  improve  training.  It  did  not 
view  it  as  a  problem,  however,  but  rather  as  a 
challenge--and  it  did  something  about  it— it 
turned  to...  AUTOMATION!  The  reasoning  appeared 
obvious-we  needed  a  better  tool.  And  we  got  the 
tool  that  everyone  was  starting  to  get  excited 
about... we  acquired  a  Computer  Aided  Instruction 
(CAI)  system.  It  seemed  patently  clear  that  the 
age  of  television  was  upon  us  and  that  the 
younger  generations  related  much  better  to  this 
medium  than  to  the  printed  word.  It  stood  to 
reason,  then,  that  students  would  1 earn  better 
and  faster  if  they  could  be  taught  by 
television.  Therefore,  we  acquired 
television-like  equipment  and  started  to  work. 

For  six  years  M-C-C-E-S  actively  developed 
CAI  lessons,  conducted  studies  and  economic 
analyses,  prepared  automated  data  processing 
(ADP)  plans,  and  so  on.  We  were  very  busy  and 
enthusiastic.  But  all  we  created  was  a  lot  of 
smoke-and  no  heat.  Nothing  happened.  We  failed 
to  produce  any  tangible  results.  However,  our 
efforts  were  not  totally  in  vain.  We  learned 
something-we  learned  what  we  did  not  want. 
Thinking  that  TV  would  somehow  equate  to 
learning,  we  had  become  enamored  with  the  hard¬ 
ware  and  had  leapt  right  into  a  hardware 
solution-without  adequately  defining  the 
problc.,1.  So  we  restudied  our  needs  and 
submitted  a  new  set  of  system  requirements  to 
the  Conwandant  of  the  Marine  Corps.  And  in  July 
of  1976,  the  Commandant  gave  the  go-ahead  to 
acquire  a  new  system  which  would  be  precisely 
tailored  to  our  needs. 

In  April  of  1977,  the  CAI  Project  Team  was 
formally  established.  It  was  a  pick-up  team, 
comprised  solely  of  in-house  personnel--no 
outside  experts  were  used.  As  such,  it  was  sort 
of  a  sand-lot  approach  to  systems  development. 
But  it  was  effective,  in  large  part  because  we 
observed  one  of  the  cardinal  rules  of  sound 
project  management- -we  dedicated  this  team 
exclusively  to  the  project  under  the  direction 
of  a  full-time  Project  Officer.  The  Project 
Team’s  first  task  was  to  develop  a  detailed  set 
of  system  specifications  based  on  the  needs 
assessment  that  had  been  submitted  to  CMC.  Two 
significant  aspects  of  this  document  were:  (1) 
it  was  expressed  in  terms  of  functional 
requirements—educational  requirements,  not 
hardware  preferences;  and  (2)  it  included  a 
requirement  for  Computer  Managed  Instruction 
(CMI).  Although  our  mind-set  at  this  time  was 
still  primarily  oriented  to  CAI,  we  had  learned 
enough  in  the  intervening  years  to  know  that  it 
was  the  total  management  of  the  educational 
process--not  merely  the  use  of  the  computer  as 
an  instructional  medium—which  held  the  greatest 
promise  of  economic  savings.  We  were  definitely 
progressing.  But  we  knew  that  we  had  much  more 
to  learn.  So,  we  devoted  1978  and  most  of  1979 


236 


system.  Site  preparation  and  coordination  with 
the  system  acquisition  agency  back  in 
Washington,  D.C.  were  its  major  activities.  As 
an  interim  measure,  necessitated  by  unexpected- 
but  typical--delays  in  the  system  acquisition 
process,  Track  1  undertook  negotiations  to  lease 
a  temporary,  low-cost  computer  system  to  serve 
as  a  training  device  for  our  unskilled  personnel 
who  would  have  to  function  as  programmers  and 
system  analysts  for  the  CBE  system  when  it  was 
installed.  Additionally,  it  would  provide  the 
much  needed  computer  support  for  development  of 
the  various  application  programs  and  for  our 
initial  endeavors  in  converting  courses  to  the 
FM/VT  strategy.  We  were  intent  on  being  ready 
to  produce  immediate  results  on  any  future  CBE 
system  as  soon  as  it  was  installed  and 
operating. 

Software-that  is,  data  base  design  and 
application  program  definition,  description,  and 
development-was  the  function  assigned  to  Track 
2.  This  was  a  most  crucial  activity  if  we  were 
to  employ  the  CBE  system  as  more  than  just  an 
instructional  device.  The  results  of  this 
effort  would  be  the  ones  which  would  yield  our 
most  iimiediate  pay-off  in  terms  of  automating 
mundane,  manual  tasks  in  the  educational  process 
and  reducing  the  requirement  for  training 
support  personnel.  Coordinating  with  the 
various  functional  sectors  of  the  school.  Track 
2  succeeded  in  developing  the  rudiments  of  our 
CBE  system.  The  six  major  subsystems  have  been 
defined  and  described  and  their  operating 
algorithms  designed--and  within  each,  the  top 
priority  applications  have  been  programmed  and 
are  operating.  One,  in  particular,  has  given  us 
our  first  glimpse  of  potential  personnel 
savings.  Once  it  is  fully  implemented,  the 
Automated  Order-writing  Program  will  enable 
MCCES  to  reduce  its  Personnel  Administration 
Center  staff  by  four  people.  A  small  beginning, 
but  a  lot  of  promise! 

Track  3  was  charged  with  the  responsibility 
of  developing  FM/VT  course  material.  It  was 
toward  this  effort  that  M-C-C-E-S  received  its 
first  outside  assistance--augmentation  of  its 
staff  by  a  Marine  Reserve  Colonel,  holder  of  a 
PhD  in  Educational  Psychology  with  over  15  years 
experience.  A  staunch  advocate  of  FM/VT,  he  has 
spearheaded  the  attack  in  this  endeavor. 

Although  courseware  development  was  ultimately 
to  become  a  collaborative  effort  between 
M-C-C-E-S  and  outside  contractors,  the 
contractual  process  involved  lengthy 
negotiations.  Unable  to  sit  idle  while  these 
transactions  were  completed,  the  School  launched 
its  own  program.  We  were  to  learn  much  in  this 
process--indeed,  our  full  perception  about  the 
FM/VT  approach  did  not  mature  until  we  had 
actually  ventured  into  the  arena  ourselves. 
During  this  effort,  we  were  also  to  revise  our 
thinking  regarding  the  role  to  be  played  by 
courseware  contractors.  We  began  on  a  small 
scale,  with  plans  for  progressively  expanded 
courseware  conversions. 

Our  first  endeavor  actually  had  its 
beginnings  during  the  Summer  of  1979  with  a 
series  of  four  experiments  involving  the 
Fundamentals  of  Digital  Logic  Course  (FDLC)--a 
two-week  block  of  instruction  at  the  tail  end  of 
our  Electronics  Fundamentals  Course.  A  small, 
easily  managed  instructional  package,  which  all 
telecommunications  and  electronics  repairmen 


have  to  take,  it  provided  the  ideal  starting 
point  for  our  FM/VT  development.  The  success  of 
the  four  experiments,  which  were  conducted 
rigorously,  with  control  groups  taught  in 
lockstep  providing  the  comparison,  prompted  the 
school  to  adopt  the  FM/VT  strategy  for  this 
sub-course  on  a  full  scale  in  December  1979. 

Since  the  FDLC  experiments  were  first  started,  a 
total  of  929  students  have  received  instruction 
under  the  FM/VT  method.  2,878  training  days  have 
been  saved  vis-a-vis  the  training  time  which 
would  have  been  required  to  train  the  same  number 
of  students  under  lockstep  instruction.  This 
averages  out  to  approximately  three  training 
days  saved  per  student.  100%  mastery  has  been 
achieved  in  all  cases,  and  there  has  been  zero 
attrition!  The  courseware  originally  developed 
by  M-C-C-E-S  has  since  been  smoothed  out  by  a 
professional  courseware  firm  and  is  now  in 
active  use. 

A  major  disadvantage  of  the  FDLC  package 
which  presently  exists  due  to  the  fact  that  it 
is  merely  a  preparatory  course,  is  that  once  it 
is  completed,  scudents  must  return  to  the 
lockstep  environment  to  pursue  their  follow-on 
courses.  Tolerating  this  for  the  time  being  in 
order  to  gain  experience  in  FM/VT  training, 
M-c-C-E-S  proceeded  to  the  next  logical 
step--development  of  an  FM/VT  course  from  which 
a  student  could  be  assigned  a  Military 
Occupational  Specialty  (MOS)  and  be  transferred 
to  the  field.  It  chose  to  attempt  this  with  the 
Ground  Organizational  Radio  Repairer  Course 
(GORRC) ,  itself  a  pilot  program  in  the  Marine 
Corps  aimed  at  redesigning  maintenance  training 
to  include  a  level  for  organizational  repair- 
men-those  limited  duty  repairmen  assigned 
to  combat  organizations.  If  conducted  in 
lockstep,  this  course  would  have  taken 
approximately  70  training  days.  However,  under 
the  FM/VT  strategy  the  average  training  time  was 
49  days--a  savings  of  21  training  days  per 
student.  A  total  of  30  students  went  through 
the  first  class  with  100%  mastery  and  no 
attritions.  The  total  training  days  saved  was 
630-just  for  the  first  iteration  of  only  one 
course.  Incidently,  the  graduates  of  the  pilot 
GORRC  have  been  sufficiently  well-received  in 
the  field  to  prompt  the  Marine  Corps  to  convene 
a  second  class  in  June  1980. 

The  third  phase  of  our  FM/VT  development 
was  to  apply  it  to  a  total  training  pipeline- 
that  is,  to  a  series  of  courses  leading  to  a 
complex  technical  MOS.  We  chose  for  tins  effort 
MOS  2841,  tne  full-blown  Ground  Radio  Repairman, 
a  perenially  short  MOS.  Recognizing  that  the 
chain  of  courses — Electronics  Fundamentals, 

Radio  Fundamentals ,  and  Ground  Radio  Repair 
coi”-ses--leading  to  this  MOS  contained  certain 
unnecessary  and  redundant  blocks  of  instruction, 
our  first  step  was  to  analyze  the  requirements 
and  redesign  the  series  into  a  single, 

"terminal"  course  of  instruction.  The  second 
step  was  to  revise  the  training  material  for  the 
FM/VT  approach.  Compa  ison  of  the  times 
required  to  train  a  single  Ground  Radio 
Repairman  reveals  that  under  the  current 
met hod--the  training  pipeline  approach--it  takes 
39  weeks.  The  redesigned,  stand-alone  course 
reduces  his  training  time  to  20  weeks— if  taught 
under  lockstep  instruction.  But--if  taught 
under  FM/VT,  it  would  take  approximately  14 
weeks-if  our  past  experience  in  achieving  a  30% 


reduction  in  training  time  is  borne  out. 
Translating  this  into  dollar  savings,  we  can 
expect  that  the  following  economies  are  to  be 
realized  for  a  single  student.  It  costs 
$6,038.00  to  train  one  student  under  the  current 
lockstep  method--this  cost  made  up  of  both  his 
salary  and  his  share  of  the  course  costs.  The 
redesigned  lockstep  course  would  cost 
approximately  $3,100.00.  But. ..if  conducted 
under  the  FM/VT  approach,  the  costs  would  dip  to 
roughly  $2,170.00.  Projecting  this  time  and 
money  savings  over  a  full  year,  training  480 
students  as  Ground  Radio  Repairmen,  we  find  that 
the  redesigned  lockstep  course  would  reflect  a 
49%  savings  in  both  training  dollars  and  student 
manyears.  The  redesigned  course  under  FM/VT 
would  reflect  a  64%  savings  in  these  costs. 

The  redesigned  lockstep  course  would,  indeed, 
reflect  a  49%  savings. . .but  it  would  still 
result  in  variable  mastery  and  high  attrition. 
The  redesigned  FM/VT  course,  however,  would 
result  in  even  greater  savings. ..and  would 
result  in  total  mastery  and  no  attrition--or 
close-to  zero  attrition. 

Simple  arithmetic  tells  you  that  multlipied 
out  over  45  courses  of  instruction  offered  by 
MCCES  for  the  6,000  plus  students  per  year,  the 
potential  savings,  in  both  time,  money  and 
manpower  is  enormous. 

The  pilot  courses  for  the  FM/VT  version  of 
the  Ground  Radio  Repairmen  now  convened  on  10 
June  1980. 

Track  4-last  of  the  implementation 
tracks— was  assigned  the  responsibility  of 
analyzing  the  Human  Factors  requirements  which 
we  had  learned  were  so  crucial  to  the  success  of 
a  computer  based  training  project.  To  be  sure, 
failure  to  adequately  address  these 
considerations  had  revealed  itself  to  be  the 
"Achilles  Heel"  of  many  other  such  endeavors. 
Paramount  among  these  were:  the  changing  role 
of  the  instructor;  the  changing  role  of  the 
student;  and  "predictive  analysis"  of  student 
learning  potential.  We  were  well  aware  of  the 
fact  that  we  could  expect  much  apprehension--and 
possibly  resistance—on  the  part  of  the 
instructor  regarding  CBE.  if  we  did  not  handle 
it  properly,  he  could  easily  begin  to  feel  that 
he  was  going  to  become  subordinate  to  the 
computer— perhaps  even  replaced  by  it.  After 
all,  the  very  nature  of  lockstep  instruction  is 
that  it  is  instructor-centered.  Therein,  the 
instructor  functions  much  like  an  actor  on  a 
stage—he  learns  his  lines  and  he  delivers  them 
from  a  stage  to  an  audience  of  students.  We 
knew  that  we  had  to  reassure  him  that  his  role 
was  not  going  to  be  diminished  or  eliminated— 
but  rather,  that  it  was  going  to  be  expanded 
and  enriched  with  a  great  deal  more 
satisfaction  than  he  had  known  before.  He  was 
no  longer  going  to  be  that  actor  on  the  stage, 
but  more  the  director  of  the  entire  production. 

We  have  been  highly  successful  in  progressively 
"transforming"  our  lockstep  instructors  into 
FM/VT  instructors  by  involving  them, 
individually  and  in  small  groups,  totally  in  the 
conversion  to  FM/VT  training.  We  have  learned 
that  "involvement"  does  not  merely  mean 
informing  them  of  what  is  being  done  or  even 
soliciting  their  opinions  and  suggestion — often 
the  furthest  extent  to  which  "participative 
management"  is  ever  pursued.  On  the  contrary, 
it  means  having  them  roll  up  their  sleeves  and 


dig  right  in  under  close  supervision.  It  means 
everything  from  designing  the  course  material, 
to  actually  serving  as  the  instructors  in  the 
FM/VT  course  they  have  worked  on,  to  finally 
evaluating  the  product  and  revising  the 
courseware  based  on  their  evaluations.  We  are 
confident  that  the  "heart  and  mind"  of  the 
instructor  will  not  be  won  by  handing  him  a  set 
of  materials  developed  by  outside  "experts"  with 
the  expectation  that  he  supportively  and 
enthusiastically  use  it.  The  "Not  Invented 
Here"  syndrome  is  a  fact  of  life  which  we  must 
fully  recognize. 

Hand-in-hand  with  the  changing  instructor 
role  is  the  new  and  different  role  of  the 
student.  Whereas  in  the  lockstep, 
instructor-centered  mode  of  instruction  he 
is--to  a  great  extent— an  anonymity  sitting  in 
the  audience,  in  FM/VT  training  he  becomes  the 
actor  in  a  student-centered  production.  Each 
actor  has  his  unique  talent — some  can  dance  and 
some  can  sing.  In  the  same  manner,  each  student 
has  his  unique  capabilities— some  are  better 
verbally,  some  have  greater  mathematical  skills, 
and  so  on.  As  it  is  the  job  of  the  director  to 
know  the  various  talents  of  his  actors  and  to 
orchestrate  them  to  obtain  the  very  best 
performance,  so  it  is  the  job  of  the  FM/VT 
instructor  to  know  the  capabilities  of  his 
students  and  manage  them  to  achieve  peak 
performance. 

To  assist  the  instructor  in  this  task,  we 
have  attempted  to  get  a  better  handle  on 
predicting  student  learning  potential.  Our 
first  experiment  with  this  took  place  with  the 
pilot  Ground  Organizational  Radio  Repairer 
Course.  Using  the  Electronic  Aptitude  (EL)  and 
General  Technical  Aptitude  (GT)  scores  from  the 
Armed  Services  Vocational  Aptitude  Battery 
(ASVAB)  test,  we  devised  a  Cognitive  Aptitude 
(C)  value  for  each  student,  scaled  from  zero  to 
3.  To  this  we  added  a  Psychomotor  Ability  (P) 
value,  based  on  a  very  rudimentary,  locally- 
developed  psychomotor  test.  And  finally,  we 
ascribed  to  each  student  an  Affective  Traits 
(A)  value  derived  from  scores  made  on  Rotter's 
Internal -External  Locus  of  Control  scale.  It 
was  theorized  that  the  A- value  would  give  some 
indication  of  a  student's  probability  of 
perservering  through  the  more  complex  course 
material  under  FM/VT  instruction.  Realizing  that 
our  measurement  instruments  were  very  crude  at 
best,  we  nevertheless  used  them  to  develop  a 
Probable  Completion  Time  (PCT)  for  each  student. 
The  results  of  this  initial  effort  are,  of  course, 
inconclusive.  Much  more  extensive  work  has  to  be 
done,  and  the  measurement  instruments  certainly 
have  to  be  much  further  refined.  However,  we 
did  show  that  the  PCTs  for  each  student  were 
fairly  accurate.  This  has  convinced  us  that  we 
can,  in  time,  become  more  adept  at  predicting  a 
student's  potential  for  mastering  a  set  of 
learning  objectives  within  a  certain  time  frame. 
Once  we  can  do  this  with  accuracy,  we  can  move 
one  step  further  toward  matching  students  to 
programs  of  instruction  that  are  most  within 
their  range  of  capabilities.  In  short,  we—as 
directors— can  better  cast  our  actors! 

In  summary,  the  Marine  Corps'  CBE  system 
being  developed  at  M-C-C-E-S  is  a  marriage  of 
three  basic  diciplines:  Educational  Strategy, 
Management  Theory,  and  Computer  Technology.  A 
review  of  its  salient  features  reveals  that  it 
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is. . .an  interactive  educational  system  based 
on  a  self-paced,  mastery-learning  strategy-- 
"J3*  1s: '  •FM/VT-  !t  emphasizes  the  importance 
of  the  instructor,  placing  the  computer  in  a 
subordinate,  supportive  role  which  offers... 
computer  data  base  management  that 
supports  the  educational  strategy. 
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DEVELOPMENT  OF  TGTS  -  A  TANK  GUNNERY  TRAINING  SIMULATOR 
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ABSTRACT 

Link-Miles  has  carried  out  a  Study  Programme  to  develop  a  Tank  Gunnery  Training  Simulator 
(TGTS).  The  Programme  was  carried  out  in  consultation  with  many  armies  worldwide,  and  in  close 
collaboration  with  a  European  army.  A  survey  of  Armour  training  methods  and  User  requirements 
was  made,  to  define  the  features  which  should  be  provided  in  the  simulator.  There  followed  an 
Engineering  investigation,  to  establish  the  design  of  a  cost-effective  device  which  meets  the 
identified  training  requirements.  This  paper  describes  the  Study  Programme,  and  the  conceptual 
design  of  the  TGTS. 


BACKGROUND 


Survey  of  User  Requirements 


The  Link-Miles  Division  of  The  Singer  Company  has 
produced  a  variety  of  simulators  representing 
tanks,  APCs,  and  other  Fighting  Vehicles  for  a 
nunber  of  armies.  These  were  all  intended  for  use 
in  the  training  of  drivers,  and  are  proving  to  be 
effective  in  that  role.  The  production  of 
Fighting  Vehicle  driver  training  equipment  brought 
the  Company  into  close  contact  with  the  Armour 
training  conmunity,  and  resulted  in  a  growing 
awareness  of  its  needs.  In  particular,  it  was 
clear  that  the  training  of  turret  crew-members  in 
the  skills  of  gunnery  presented  problems  in  view 
of  escalating  ammunition  costs  and  the  tighter 
restrictions  placed  upon  the  availability  of 
practice  ranges.  These  difficulties  are 
exacerbated  by  the  introduction  of  higher-cost  and 
longer-range  anmunition.  In  addition,  there  is 
the  ever  present  need  to  achieve  and  maintain  a 
high  level  of  coribat  readiness,  which  requires 
that  crews  should  frequently  exercise  their  skills. 

It  appeared  to  Link-Miles  that  existing  methods 
and  equipment  used  in  turret  crew  training  were 
not  able  to  support  the  training  which  armoured 
units  would  require  to  carry  out  in  the  future. 

The  Company  therefore  resolved  to  develop  new 
equipment  to  support  tank  gunnery  training  in  a 
cost-effective  manner,  and  initiated  a  Study 
Programme  to  that  end. 


THE  STUDY  PROGRAMME 

The  Study  Programme  followed  a  logical  plan,  as 
shown  in  Figure  1,  from  an  initial  survey  to 
determine  the  requirements  of  the  market  to  the 
production  of  a  Proposal  for  prospective  customers. 

The  Study  Programme  had  three  objecti ves  :- 

(1)  To  establish  the  training  which  could  be 
achieved  in  a  Tank  Gunnery  Training  Simulator 
(TGTS),  and  define  the  simulator  features 
necessary  to  facilitate  such  training. 

(2)  To  investigate  the  most  cost-effective 
technical  methods  of  providing  the  required 
simulator  features. 

(3)  To  establish  the  conceptual  design  of  a 
simulator  which  meets  the  defined 
requirements. 


When  developing  a  new  product  there  are  two  traps 
into  which  the  unwary  may  fall.  Firstly,  they  may 
indulge  in  "armchair  philosophising",  and  imagine 
that  they  already  know  what  the  requirements  of 
the  potential  user  will  be.  Unless  the  members  of 
the  development  team  are  experienced  armour  crew¬ 
men  as  well  as  training  specialists  and  engineers, 
it  is  very  likely  that  the  perceived  requirements 
will  be  distorted,  incomplete,  and  lay  emphasis 
upon  the  wrong  aspects  of  training.  Secondly,  if 
the  needs  of  one,  and  only  one,  army  are  explored 
in  depth  then  a  solution  to  them  may  indeed  be 
reached.  However,  armies  vary  considerably  in  the 
nature  and  equipment  of  their  tanks,  the  quality 
and  number  of  soldiers  to  be  trained,  and  the 
resources  at  their  disposal.  Hence  any  conclusions 
reached  after  studying  a  single  army  "ay  well  be 
so  specific  to  their  needs  that  extrapolation  to 
other  users  would  not  be  possible. 

For  these  reasons,  the  Link-Miles  survey  was  based 
upon  a  sample  of  about  ten  armies,  spread  world¬ 
wide.  They  were  chosen  to  reflect  different  tank 
types,  soldiers  (volunteer  or  conscript),  and 
nature  of  resources.  All  the  armies  approached 
willingly  entered  into  discussions,  and  without 
exception  were  extremely  helpful  to  the  project. 

The  survey  team  consisted  of  staff  with  expertise 
in  the  areas  of  training  psychology,  simulation 
engineering,  optics,  and  related  disciplines.  A 
large  number  of  visits  were  made  to  training 
schools,  technical  establishments,  and  firing 
ranges.  In-depth  discussion  with  staff  at  all 
levels  gave  an  insight  into  both  overall  training 
philosophies  and  everyday  practical  problems. 
Manuals  and  training  aids  were  made  available  to 
the  team,  who  were  thus  better  enabled  to  under¬ 
stand  the  intricacies  of  each  army's  equipment. 

An  especially  close  relationship  developed  with 
one  European  army.  Mentrers  of  the  Link-Miles  team 
were  able,  by  courtesy  of  this  army,  to  undergo 
training  in  all  turret  crew  functions  in  a  manner 
similar  to  that  employed  for  recruits.  This 
training  was  over  an  extended  period,  and  included 
live  firing  exercises  on  the  range  using  a  variety 
of  ammunition  types.  This  first-hand  knowledge 
and  experience  of  crew  functions  and  the  finer 
points  of  gunnery  technique  was  invaluable  in 
determining  cost-benefit  trade-offs  and  putting 
into  perspective  the  many  ideas  and  comments 
received  during  the  survey  phase. 
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The  team  paid  special  and  detailed  attention  to  all 
tasks  that  the  turret  crew  are  required  to  carry 
out  in  preparing  to  fire  the  tank  cannon,  searching 
for  and  acquiring  targets,  and  engaging  them  with 
appropriate  gunnery  technique.  The  skills  employed 
at  each  stage  of  the  operation  were  identified,  and 
analysed  in  terms  of  training  requi rements .  These 
requirements  were  then  compared  with  the  methods 
currently  in  use  for  training  them,  and  so  areas  in 
which  synthetic  training  would  be  effective  were 
identi  fied. 

Thus,  from  the  survey  phase  there  emerged  the 
concept  of  a  simulator  which  could  be  used  to 
facilitate  gunnery  training,  and  an  outline  of  the 
attributes  which  it  must  have  to  be  successful. 

This  outline,  and  the  accumulated  experience  of 
the  team,  was  carried  forward  to  the  next  phase  of 
the  programme. 


that  a  configuration  which  permitted  training  in 
some  aspects  of  operation  would  compromise 
efficiency  when  teaching  other,  more  essential 
tasks.  It  was  therefore  decided  that  the  simulator 
was  to  provide  a  means  for  effective  training  of 
the  turret  crew  to  be  skilled  in  the  gunnery  task, 
when  using  the  tank  cannon.  It  should  be  possible 
to  train  recruits  in  handling  all  equipment 
concerned  with  gunnery,  and  to  work  rapidly  as  a 
crew  when  engaging  an  enemy  target.  The  level  of 
proficiency  attained  in  such  a  simulator  should  be 
at  least  as  great  as  the  level  attained  with  firing 
on  a  range.  The  simulator  should  also  provide 
practice  for  trained  crews,  to  maintain  their 
proficiency.  It  is  important  to  note  that  the 
TGTS  was  deliberately  configured  to  address  the 
problem  of  basic  gunnery  training,  and  not  those 
of  battle  tactics  or  co-operation  between  the 
turret  crew  and  driver. 


Performance/Cost  Trade-Off  Analysis  &  Specification  Crew  to  be  Trained 


Figure  1  shows  the  way  in  which  the  Study  Programme 
was  structured.  The  team  commenced  work  on  the 
programme  with  no  pre-concei ved  ideas  of  the 
training  device  configuration,  nor  even  any  stock 
techniques  which  it  wished,  for  commercial  reasons, 
to  include  in  the  TGTS.  Hence  design  work 
proceeded  unfettered  by  any  restrictions  other  than 
training  utility,  cost-effectiveness,  and  good 
engineering.  The  final  design  evolved  gradually, 
with  a  series  of  judgements  on  engineering 
feasibility,  cost  of  competing  techniques,  and 
their  impact  upon  the  overall  effect  of  the 
teaching  environment.  It  was  often  found  that  a 
decision  in  one  area  affected  other  areas  of  the 
simulator  -  areas  which  had  already  been 
tentatively  formulated. 

In  due  course,  however,  an  appropriate  balance  was 
struck  between  all  factors. 


SURVEY  OF  USER  REQUIREMENTS 

i 

PERFORMANCE /COST  TRADE-OFF  ANALYSIS 

* 

SPECIFICATION 

* 

PROPOSAL 


FIGURE  1  -  TGTS  STUDY  PROGRAMME 


Scope  of  Simulator  Training 

A  decision,  critical  to  the  remainder  of  the 
programme,  was  made  concerning  the  scope  of  that 
training  around  which  the  simulator  should  be 
configured.  A  danger  here  was  that  any  attempt  to 
design  a  device  which  would  support  training  of 
all  tank  crew-menters  in  the  full  range  of  combat 
duties  would  inevitably  result  in  a  prohibitively 
expensive  piece  of  equipment.  It  was  also  clear 


Because  of  the  very  close  interaction  between  the 
cotmtander  and  gunner,  it  was  judged  essential  that 
the  simulator  should  include  positions  for  both 
these  crew-members.  These  positions  should  permit 
the  teaching  of  all  gunnery  activities,  including 
the  facility  for  the  commander  to  aim  and  fire  the 
gun  himself. 

The  question  of  whether  or  not  to  include  the 
loader  in  the  simulation  was  more  difficult.  It 
appeared  that  adequate  training  of  loaders  could 
be  achieved  in  other  part-task  trainers,  and  that 
the  physical  presence  of  that  crew-member  in  the 
simulator  was  not  absolutely  necessary.  An 
alternative  school  of  thought,  however,  was  that 
the  presence  of  the  loader  may  be  worth  the  cost 
increment  caused  by  adding  his  position  in  the 
simulator,  complete  with  an  operable  breech  and 
loadable  rounds.  It  was  therefore  determined  that 
the  loader  would  not  be  included  in  a  basic 
simulator,  but  that  he  would  be  himself  simulated 
by  software  and  audio  generation  devices.  A 
loader  station,  with  all  training  facilities, 
would  however  be  offered  as  an  optional  extra  to 
those  customers  who  wish  to  train  a  complete 
turret  crew  and  are  prepared  for  the  additional 
cost  of  so  doing. 


FEATURES  OF  THE  SIMULATOR 

The  specification  which  emerged  from  the  cost- 
benefit  trade-off  is  summarised  in  Table  1.  It 
will  be  seen  that  there  are  a  few  optional  items, 
as  discussed  above,  to  cater  for  the  requirements 
of  individual  armies  which  have  specific  training 
needs.  It  is  believed,  however,  that  the  basic 
specification  reflects  a  distillation  of  the 
requirements  of  many  different  armies. 

When  the  simulator  features  had  been  broadly 
defined,  the  team  proceeded  to  consider  the 
engineering  methods  by  which  they  may  be 
implemented.  Cost-benefit  trade-off  continued,  to 
ensure  that  optimum  hardware  solutions  were  reached. 

An  artist's  impression  of  the  simulator  which 
emerged  from  this  phase  of  the  programme  is  shown 
in  Figure  2.  The  major  components  of  the 
simulator  are  discussed  below. 
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TABLE  1  -  SIMULATOR  FEATURES 


FACILITY 

BASIC 

OPTION 

Crew  to  be  trained 

Commander  and  Gunner 

Loader 

Types  of  Shooting 

Day  (Variable  Visibility) 

Dusk 

Night 

Targets 

Up  to  4  simultaneously,  fixed  or  moving 

Rangefinder 

Laser 

Coi  ncident 

Stereoscopic 

Stadi ametric 

Firing-on-the-move 

Visual  effects  of  own  tank  motion 

Observation  of  fall 
of  shot 

Trajectory  of  tracer 

Impact  effects 

Ri  cochet 

Trajectory 

Disturbances 

Lateral  Wind 

Ballistic  Dispersion 

Tank  Cant 

Observati  on 

Disturbances 

Turret  Recoil 

Gun  Flash 

Gun  Smoke 

Controls 

All  controls  available  to  commander  and  gunner 
(normal  and  reversionary) 

Loader  station 
controls  & 
breech 

Terrain  and  Target 
Viewing 

Commanders  Head  Up  View  (di rect/ through 

Binoculars) 

All  Cotmiander's  Aiming  Optics  and  Forward 

Vision  Block 

All  Gunner's  Aiming  Optics  and  Vision  Block 

All  Night  Vision  devices 

Loader's  Vision 
Block 

Commander's  Head- 
Up  Field  of  View 

120°  Wide  by  30°  High.  Central  60°  Wide 
sector  is  targetable 

Simulated  Range 

800  -  3000  Metres 

Sound  Effects 
(examples) 

Breech  Closing  and  Loader’s  Call 

Turret  Rotation 

Gun  Elevation 

Gun  Blast,  and  Ejection  of  Shell  Case 

Background 

Battle  Sounds 

Communications 

Tank  Intercom 

Instructors  to  Crew 

Secure  Inter-Instructor 

Background 

Radio  Chatter 

Instructor 

Facilities 

View  Targetable  Terrain  Scene 

Repeat  of  Commander's  and  Gunner's  view  through 
aiming  optics,  and  Fall  of  Shot  observation 

Monitor  Crew's  Principal  Controls  and 

Read  Outs 

Hit-Miss  Read-Out  on  VDU  and  in  Hard-Copy 
Malfunction  Introduction  capability 

Introduce  trajectory  disturbances 

Pre-programmed  Exercises 

Record-playback 
of  view  through 
aiming  optics 
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FIGURE  2  -  ARTIST'S  IMPRESSION  OF  TGTS 


The  Visual  System 

The  most  important  and  demanding  aspect  of  the  TGTS 
is  the  visual  system,  by  means  of  which  targets, 
and  the  terrain  in  which  they  are  located,  are 
presented  to  the  gunnery  crew.  Thorough 
consideration  was  given  to  all  possible  systems, 
including  three-dimensional  model  boards,  computer¬ 
generated  imagery,  photographic  film,  and 
television  based  systems.  V.  is  vital  to  recognise 
the  fact  that  current  technology  does  not  allow  the 
level  of  visual  simulation  to  reproduce  totally  the 
real-world  environment.  Consequently,  it  became 
essential  to  identify  the  critical  visual  system 
performance  requirements  necessary  to  meet  the 
training  needs  of  the  TGTS. 

It  had  been  established  during  the  survey  phase 
that  users  need  to  detect  and  identify  targets  in 
the  TGTS  at  the  same  range  as  can  be  achieved  in 
the  actual  tank.  This  enabled  the  team  to  determine 
the  minimum  resolution  of  the  visual  system,  so 
that  for  example  with  7  -  power  binoculars  a  2.3 
metre  high  target  may  be  detected  at  a  range  of 
3,000  metres  and  identified  by  type  at  2,000 
metres.  This  requires  the  targets  to  be  displayed 
at  a  high  resolution. 


To  prevent  a  noticeable  difference  in  quality 
between  targets  and  terrain  -  which  would 
effectively  make  the  targets  stand  out  -  the 
terrain  image  must  also  be  displayed  at  high 
resolution.  Finally,  from  the  survey  and 
examination  of  existing  training  devices,  it  was 
clear  that  both  targets  and  the  terrain  scene  must 
be  realistic.  Any  simplified  presentation,  whilst 
possibly  justifiable  on  the  grounds  of  strict 
analysis  of  training  requirement,  is  rapidly 
dismissed  by  soldiers  as  a  "shooting  gallery",  and 
valuable  motivation  is  soon  lost-  Also,  the 
presentation  of  realistic  terrain  enables  training 
to  be  given  in  ground  surveillance  techniques,  and 
is  a  valuable  additior  to  the  simulator. 

In  view  of  all  these  factors,  the  system  chosen 
was  based  upon  the  projection  of  large-format  high 
resolution  photographic  transparencies.  Such  a 
system  has  the  following  advantages :- 

(1)  Real-world  scene  content,  allowing  both  head- 
up  viewing  and  through  all  optical  devices 
simultaneously.  The  resolution  obtained  is 
better  than  1  arc  minute  at  the  unaided  eye. 

(2)  Increased  scene  content  when  viewed  through 


magnifying  sights . 

(3)  Good  and  realistic  colour  reproduction. 

(4)  Very  bright  images,  as  powerful  projectors 
may  be  used. 

(5)  Effects  of  restricted  visibility  (fog,  mist) 
are  readily  and  realistically  obtained. 

(6)  Night  scenes  can  be  reproduced,  using  the 
appropriate  transparencies.  By  special 
techniques,  the  effects  of  white  light  or  I.R. 
searchlights  can  be  produced,  and  thermal  or 
low-light  TV  viewing  systems  employed  for 
gunnery. 

(7)  Realistic  target  images,  matched  in  quality 
to  the  terrain.  Targets  may  be  tanks,  APCs, 
trucks,  etc.  They  may  be  momentarily  flashed 
at  a  brighter  level,  under  instructor  control, 
to  call  attention  to  a  particular  target  for 
engagement. 

(8)  Real-world  target  detection  ranges  are 
achieved. 

(9)  One  design  of  visual  system  may  be  used  for 
head-up  viewing  and  all  types  of  sights  on 
any  model  of  tank.  There  is  no  need  to  re¬ 
design  for  each  tank. 

(10)  A  single  visual  system  will  serve  head-up 
viewing,  all  vision  blocks,  and  all  sights  - 
simultaneously. 

(11)  Terrain  scenarios  are  readily  changed  by  the 
instructor. 

In  the  Link-Miles  design,  three  high-powered 
projectors  are  used  to  project  the  images  of  large 
format  slides  onto  a  screen  which  is  a  segment  of 
a  sphere.  The  images  are  matrixed  to  give  a  field 
of  view  120°  wide  and  30°  high.  Rapid  auto-change 
facilities,  for  8  slides  per  projector,  are 
provided  to  enable  different  scenes  to  be 
presented  to  trainees  as  required. 

Target  images  are  produced  by  four  smaller 
projectors,  which  may  be  operated  independently. 

The  target  slides  are  selectable  from  magazines, 
scaled  for  range,  and  projected  upon  the  central 
60°  segment  of  the  screen.  The  target  images  are 
highly  realistic,  and  acceptable  to  observers. 

The  targets  may  move  under  computer  control . 
Programming  facilities  are  included  to  ensure  that 
moving  targets  appear  to  be  obscured  as  they  pass 
behind  features  on  the  terrain  scene. 

All  projectors,  both  terrain  and  target,  are 
mounted  on  a  large  moveable  platform  supported  on 
a  gantry  above  the  simulated  turret.  This  plat¬ 
form  is  caused  to  move,  for  three  purposes 

(1)  To  give  the  illusion  of  turret  rotation.  The 
simulated  turret  remains  stationary,  and  the 
visual  scene  rotates  relative  to  it.  Although 
unusual,  this  arrangement  gives  a  good  motion 
cue  without  the  expense  of  actually  rotating 
the  crew  compartment.  (Movement  of  sights  in 
elevation  is  achieved  by  actual  movement  of 
the  sight,  as  in  the  real  tank). 


(2)  To  give  the  illusion  of  whole  tank  motion 
when  firing  on-the-move.  When  firing  on-the- 
move  the  most  important  training  task  is  to 
use  the  stabiliser  and  null  out  residual 
motion  of  the  sight  graticules. 

From  the  survey,  it  is  believed  that  there  is 
little  training  benefit  from  actually  moving 
the  firing  point  through  the  simulated  terrain. 
Therefore,  in  the  TGTS,  the  simulated  firing 
point  will  remain  stationary  and  the  visual 
effects  of  motion  will  be  produced,  together 
with  signals  to  drive  the  stabiliser  system. 

In  addition,  a  small  vibrator  will  be  fitted 
to  each  crew  seat  to  provide  the  jolts  and 
disturbances  which  occur  when  the  tank  is 
moving,  and  which  can  affect  shooting  accuracy. 

(3)  To  give  the  illusion  that  the  tank  is  canted, 
when  stationary. 

Additional  equipment  will  be  used  to  provide 

special  visual  effects  when  the  gun  is  fired:- 

(1)  A  flash  generator  will  simulate  the  optical 
effects  of  muzzle  flash. 

(2)  Variable  filters  will  give  the  impression  of 
transitory  muzzle  smoke. 

(3)  An  eye-safe  laser  will  depict  the  tracer  of 
the  projectile.  This  will  fall  according  to 
an  accurate  trajectory,  and  will  behave  with 
respect  to  the  target  according  to  the 
accuracy  or  otherwise  of  gun  lay.  For  example, 
it  will  occult  behind  the  target  for  an  over¬ 
shoot,  will  fall  short  correctly,  or  miss  the 
target  laterally.  The  laser  will  also  be  used 
to  depict  the  impact  flash  of  a  hit  on  target. 

(4)  Burst-effect  projectors  will  be  used  to  produce 
other  visual  effects  of  target  hit  or  miss, 
such  as  a  pillar  of  smoke  with  HESH  ammunition. 

As  a  result  of  (3)  and  (4),  projectile  impact 
effects  may  be  observed,  and  corrections  to 
aim  made  using  the  gunsight  graticules. 


Range-Finding  and  Fire  Control 

As  a  result  of  the  wide-ranging  survey,  Link-Miles 
is  aware  of  the  variety  of  different  range-finding 
and  Fire  Control  systems  in  use.  The  precise  type 
fitted  to  each  model  of  tank  simulated  will  of 
course  be  reflected  in  the  TGTS  for  that  tank. 
Link-Miles  has  design  solutions  for  all  the  well- 
known  systems  of  range- finding 

(1)  Laser 

(2)  Coincident 

(3)  Stereoscopic 

(4)  Stadiametric 

The  coincident  and  stereoscopic  systems  in 
particular  presented  difficult  design  problems, 
but  a  simulation  technique  has  been  evolved  which 
gives  realistic  optical  effects  and  a  real-world 
distribution  of  ranging  measurements  when  used  by 
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gunners. 

All  the  facilities  of  on-board  Fire  Control 
Computers  are  useable  in  the  simulator. 


Weapon  System  Simulation 

A  key  requirement  of  the  weapon  system  simulation 
is  that  it  should  be  extremely  accurate,  so  that 
hits  or  misses  are  a  true  reflection  of  gun  lay. 
Anything  less  would  result  in  a  lack  of  face 
validity  of  the  T6TS  at  best,  and  negative  training 
at  worst. 

In  order  to  achieve  this,  it  is  necessary  to 
compute  the  relationship  of  the  projectile  in 
flight,  the  target,  and  the  terrain.  The 
computations  take  place  in  a  digital  computer, 
which  also  deals  with  other  system  functions.  Each 
terrain  scene  is  digitised,  and  the  information  is 
stored  in  the  computer  data  base. 

When  the  cannon  is  fired,  the  computer  continuously 
solves  the  ballistic  equation  of  the  specific 
ammunition  used,  under  the  simulated  conditions 
(wind,  temperature,  etc),  set  by  the  instructor. 

The  output  of  this  computation  is  used  to  position 
the  laser  spot,  representing  the  tracer,  and  to 
decide  the  projectile  point  of  impact.  Upon  impact, 
the  simulation  system  determines  whether  or  not  the 
target  has  been  hit,  and  activates  appropriate 
effects  on  the  visual  display.  The  system  is  able 
to  achieve  real-world  accuracy,  and  to  calculate 
miss  distance.  This  information  is  automatically 
presented  to  the  instructor,  who  is  thus  relieved 
of  the  necessity  to  make  subjective  decisions 
based  upon  his  own  observations. 


The  Simulated  Turret 

Because  of  the  confined  space  in  a  real  turret,  and 
the  restraints  placed  upon  movement  and  access¬ 
ibility  of  equipment,  it  was  considered  necessary 
to  achieve  a  realistic  replica  of  the  turret 
interior.  All  controls  and  displays  that  are 
relevant  to  gunnery  or  communications  will  be 
present,  and  either  functional  replicas  or  actual 
Government  Furnished  Equipment.  Thus  the  trainees 
will  be  enabled  to  build  up  fast  and  accurate 
movement  patterns,  which  will  transfer  directly  to 
the  controls  and  viewing  devices  of  the  real  tank. 

The  simulated  turret  will  be  fabricated  from  normal 
engineering  materials,  and  will  be  much  lighter  in 
weight  than  the  armoured  turret  of  the  vehicle. 

It  will,  however,  be  strongly  built  for  two 
reasons 

(1)  During  the  survey,  the  Link-Miles  team 
observed  that  soldiers  expect  their  equipment 
to  be  rugged.  Simulated  tanks  are  not 
expected  to  encounter  the  hazards  of  confcat, 
but  they  are  likely  to  spend  many  thousands 
of  hours  at  the  mercy  of  hands  and  bootsl 

(2)  In  the  actual  tank,  the  post-firing  recoil  of 
the  turret  is  a  major  cue  to  successful 
completion  of  the  firing  sequence.  In 
addition,  if  this  cue  were  omitted  from  the 
simulator  there  would  be  a  danger  of  poor 
habits  being  formed,  as  the  crew  would  have 


no  incentive  to  brace  against  the  brow  pads 
prior  to  firing.  For  these  reasons,  the  T6TS 
turret  will  recoil  upon  firing,  being  moved 
rapidly  backwards  for  a  short  distance.  Cost 
benefit  trade-offs  in  this  area  resulted  in 
the  recoil  cue  being  limited  to  3g. 


Aural  Simulation 

As  every  Armour  crewman  knows,  tanks  are  noisy 
places  of  work.  Some  of  the  sounds  heard  are 
irrelevant  to  training,  yet  others  give  useful, 
sometimes  essential,  cues  to  the  status  of  equip¬ 
ment.  The  TGTS  will  have  an  aural  simulation 
system  based  mainly  upon  electronic  synthesis 
techniques.  The  precise  sounds  to  be  simulated 
will  depend  entirely  upon  the  tank  which  it  is 
desired  to  replicate.  Some  examples  are:- 

(1)  Turret  traverse  in  powered  mode 

(2)  Gun  elevation  in  powered  mode 

(3)  Fume  extractor  blower 

(4)  Gun  blast  and  shell  case  ejection 

When  a  human  loader  is  not  present  in  the  simulator, 
additional  systems  will  produce  the  sound  of  the 
breech  closing  and  the  loader's  call,  on  the 
intercom,  that  he  is  ready  for  firing. 

Aural  cues  will  be  injected  into  the  crew  headsets, 
which  will  also  be  employed  for  the  normal  intercom 
function.  Levels  will  be  chosen  to  equate  with 
those  in  the  tank,  with  the  exception  of  gun  blast. 
This  latter  will  be  set  somewhat  lower  than  in  the 
tank,  to  give  good  cueing  without  the  health  risk 
of  cumulative  noise-induced  injury.  In  this 
context,  it  is  worth  recalling  that  the  simulator 
may  well  be  used  to  give  each  crewman  many  more 
shots  than  he  could  be  allowed  on  a  real  range. 

Hence  he  could  be  exposed  to  more  noise  events  in 
the  TGTS. 

Aural  cueing  will,  of  course,  be  under  instructor 
control  so  that  a  teaching  point  can  be  made  when 
required  without  competing  for  attention  with  the 
simulator  itself. 


Instructor  Facilities 

Careful  consideration  was  given  to  the  way  in  which 
facilities  for  the  simulator  instructor  could  be 
arranged.  The  philosophy  adopted  was  that  every 
effort  must  be  made  to  ensure  the  operation  and  use 
of  TGTS  is  as  simple  and  self-explanatory  as 
possible.  This  would  minimise  the  time  spent  in 
purely  simulator  -  related  operations,  and  maximise 
the  time  which  the  instructor  has  to  teach  and 
evaluate  his  trainees. 

The  TGTS  is  a  little  unusual  in  that  there  are  two 
instructor  stations,  although  it  may  be  operated 
by  either  one  or  two  instructors  as  desired. 

It  is  recommended  that  each  TGTS  should  have  at 
least  one  permanent  instructor,  who  is  thoroughly 
familiar  with  its  operation  and  is  able  to  employ 
all  the  facilities.  This  will  ensure  that  the 
potential  of  the  device  is  fully  exploited,  and 
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maximum  savings  of  real  ammunition  are  made.  This 
instructor  is  located  at  a  small  look-over  console 
placed  immediately  beside  the  simulator,  as  shown 
in  Figures  2  and  3.  The  console  has  complete 
facilities  for  the  setting  up  and  control  of 
exercises,  which  may  be  set  up  and  varied  in  real 
time  or  used  in  a  pre-programmed  mode.  It  is 
envisaged  that  the  pre-programmed  mode  will  be 
used  primarily,  with  a  large  number  of  exercises 
stored  in  the  simulation  computer.  Activation  of 
a  particular  exercise  -  chosen  from  the  syllabus 
of  training  in  a  progressive  manner  -  will  set  up 
the  terrain  upon  which  shooting  is  to  take  place. 
Pre-planned  targets  will  then  appear  on  the  terrain 
in  appropriate  positions,  stationary  or  moving,  at 
programmed  times  from  exercise  start.  The 
instructor  will  thus  be  relieved  of  set-up  chores, 
and  some  degree  of  standardised  training  will  be 
achieved.  The  instructor  will  be  provided  with  a 
variety  of  monitoring  devices,  by  means  of  which 
he  may  check  the  actions  and  progress  of  his 
trainees.  The  most  important  of  these  is  a  visual 
repeat  of  the  "through  the  sights"  view,  with 
graticule  and  all  visual  effects  as  seen  by  the 
turret  crew.  This  enables  the  instructor  to 
evaluate  gun  aiming  techniques  used  by  the 
trainees.  He  does  not,  however,  use  this  system 
to  decide  if  the  projectile  hit  or  missed  the 
target  -  that  is  determined  and  signalled  by  the 
simulator,  which  gives  miss  distances  plus  the 
corrections  required  to  achieve  a  hit  on  target. 

The  information  is  presented  to  the  instructor  on 
a  Visual  Display  Unit,  and  is  also  available  in 
hard  copy  for  debrief  purposes.  Also  for  debrief 
it  is  possible  to  use  a  video  tape  recorder,  which 
records  the  "through  the  sights"  view. 


FIGURE  3  -  LOCATION  OF  INSTRUCTORS  AND  TRAINEES 


A  second,  on-board  position  for  the  instructor  is 
provided  at  the  rear  of  the  simulated  turret,  as 
shown  in  Figure  3.  This  position  enables  close 
physical  contact  with  the  trainees  when  necessary, 
and  has  very  basic  controls  for  activation  of 
exercises  plus  a  "through  the  sights"  repeat 
display.  It  is  planned  that  this  position  should 
be  occupied  by  the  squad  instructor  assigned  to 
the  crew  under  training,  so  that  he  can  assist  in 
the  early  stages  of  learning  to  operate  the  turret 
equipment.  Use  of  this  station  will  clearly  depend 
upon  the  philosophy  of  each  particular  army.  It 
has  therefore  been  configured  as  a  module,  and  may 
be  omitted  from  the  TGTS  if  so  required. 


CONCLUSION 

It  is  difficult  to  adequately  summarise  the  work 
of  several  years  in  a  relatively  brief  paper  such 
as  this.  Attention  has  therefore  been  paid  to  the 
more  important  issues,  and  has  of  necessity  omitted 
or  glossed  over  much  of  the  detailed  analysis  and 
system  design.  The  final  stage  of  the  programme, 
that  of  producing  a  Proposal,  has  now  been  reached. 
Link-Miles  is  therefore  in  a  position  to  offer 
specific  Proposals,  based  on  the  type  of  tank 
operated,  to  potential  customers. 

In  conclusion,  it  is  worth  summarising  the  most 
significant  features  of  the  TGTS:- 

(1)  The  TGTS  enables  tank  crews  to  be  trained  in:- 


(a) 

Target  detection 

W 

Target  identification 

(c) 

Range-finding 

(d) 

Gun  aiming 

(e) 

Tracking  a  moving  target 

(f) 

Tracking  when  own  tank  is 

moving 

(9) 

Gun  fi  ri  ng 

(h) 

Observation  of  projectile 

impact  effects 

(i) 

Correction  of  aim 

(2)  The  design  is  readily  adaptable  to  different 
types  of  tank. 


(3)  The  design  is  suited  to  all  range-finding  and 
Fire  Control  Systems,  and  real-world  ranging 
and  shooting  accuracies  are  achieved. 


(4)  Head-up  viewing  and  use  of  binoculars  is 
provided  for. 


(5)  All  vision  devices  and  sights  can  be  used 
simultaneously. 


(6)  Visual  system  gives  a  realistic  scene  with 
high  brightness  and  resolution.  Field  of 
view  is  120°  wide. 
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(7)  Up  to  4  targets  are  displayable  simultaneously, 
stationary  or  moving,  with  a  wide  and  rapid 
choice  of  type. 

(8)  Comprehensive  and  realistic  sound  simulation. 

(9)  Turret  recoil  during  firing  is  simulated. 

(10)  Comprehensive,  easy  to  use  instructor's 
facilities,  organised  for  maximum 
concentration  on  the  teaching  task  and 
standardisation  of  training.  Removal  of 
siijective  elements  in  assessment  of  shooting 
accuracy.  Either  one  or  two  instructors  may 
be  employed. 

(11)  Training  of  essential  elements  in  firing  on- 
the-move. 

(12)  All  ammunition  types  can  be  fired,  with 
appropriate  and  realistic  effects  for  each 
one. 

(13)  Tracer  behaves  realistically,  including 
occulting  behind  the  target  in  the  event  of 
an  overshoot. 

(14)  Both  day  and  night  firing,  using  all 
appropriate  daylight  and  special  vision 
devices,  with  variable  visibility. 

(15)  Simulation  is  based  on  digital  computation, 
giving  overall  accuracy  as  found  in  the  tank. 


(16)  Use  of  the  TGTS  will  enable  armies  to  save 
costs  in  the  following  areas 

(a)  Reduction  in  the  need  to  fire  live 
ammunition  during  training. 

(b)  Reduced  demand  for  training  ranges. 

(c)  Reduced  fuel,  maintenance,  and  tank 
replacement  costs  due  to  lower  useage. 

(d)  Less  logistic  support  required  for 
remote  training  areas. 

(17)  The  TGTS  can  be  located  at  the  training 
school  or  Unit  HQ,  and  is  available  for  use 
at  any  time,  without  limitation  due  to  time 
of  day  or  week,  weather,  or  availability  of 
consumable  stores. 

(18)  The  TGTS  is  a  controllable  training 
environment.  Graded  exercises  may  be  carried 
out  with  close  supervision  by  an  instructor 
who  has  better  information  on  trainee's 
actions  than  he  has  on  a  live  firing  range. 

It  is  therefore  anticipated  that  training 
time  will  be  reduced  and  standards  improved. 
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ABSTRACT 

The  ability  of  today's  surface  Navy  to  carry  out  its  mission  depends  not  only  on  advanced 
weapons  systems  and  highly  sophisticated  electronics  countermeasures,  but  also  on  the 
reliability  of  the  steam  propulsion  plants  which  generate  electrical  power  and  propel  the 
ships  around  the  world.  Reliability  of  a  propulsion  plant  depends  upon  a  crew,  both  enlisted 
men  and  officers,  who  are  well  trained  in  equipment  maintenance  and  the  ability  to  operate 
the  plant  safely  both  under  normal  and  casualty  conditions.  To  achieve  this  required  ex¬ 
pertise  through  training  is  the  function  of  the  1200  PSI  Propulsion  Plant  Trainer,  Device 
19E22. 

This  paper  will  discuss  how  the  1200  PSI  Propulsion  Plant  Trainer  executes  its  training 
mission  through  the  following  major  elements: 

1.  A  full-scale  mockup  of  the  engineering  spaces  including: 

(a)  Engineer  Room 

(b)  Fire  Room  and  Forced  Draft  Blower  Room 

(c)  Auxiliary  Machinery  Spaces,  consisting  of  Auxiliary  Machinery  Room  No.  1, 
Auxiliary  Machinery  Room  No.  2,  and  Electrical  Central. 

2.  A  high-speed  digital  computer  and  associated  Real  Time  Interface  and  peripheral 

equipment. 

3.  Computer  programs  including: 

(a)  Executive  program 

(b)  Math  Model  programs 

(c)  Input/Output  programs 

4.  Instructor  Station 


BACKGROUND 

The  days  of  the  World  War  II  stick  shift  de¬ 
stroyer  are  long  gone.  Technological  advances  and 
changes  in  mission  requirements  during  the  post 
World  War  II  period  have  brought  about  a  new  breed 
of  ships  to  the  fleet.  Today's  surface  Navy  re¬ 
quires  the  use  of  advanced  weapons  systems  and 
sophisticated  electronics  to  carry  out  its  mission. 
Larger  ships  with  higher  power  generation  and 
propulsion  capabilities  are  needed  to  accomodate 
and  support  these  complex  systems  and  today's 
possible  tactical  situations.  The  1200  PSI  steam 
propulsion  and  power  generation  plant  is  one  of 
the  high  capacity  systems  in  use  today  on  many 
classes  of  ships.  This  plant  has  a  far  greater 
steam  generating  capability.  Increased  flow  rates, 
and  generates  steam  pressures  nearly  twice  that 
used  on  World  War  II  ships.  As  a  result,  human 
responses  were  no  longer  adequate  to  control  and 
regulate  these  plants  and  had  to  give  way  to 
sophisticated  automated  control  systems.  In  turn. 


this  high  capacity  and  complexity  of  the  1200  PSI 
plant  necessitated  the  generation  of  new  and  im¬ 
proved  maintenance  and  repair  techniques,  as  well 
as  the  development  of  skilled  and  properly  trained 
personnel  to  operate  and  maintain  it. 

Initially  the  1200  PSI  plant  had  poor  reliabil¬ 
ity  record,  with  numerous  equipment  failures, 
casualties,  personnel  injuries  and,  regretfully, 
deaths,  thus  severely  affecting  the  ability  of  the 
ships  with  these  plants  to  perform  this  mission. 

A  study  revealed  that  much  of  the  problem  was  due 
the  lack  of  proper  training  in  plant  maintenance 
and  operational  procedures.  To  correct  this  situa¬ 
tion,  a  1200  PSI  "Hot  Plant"  was  constructed  at  the 
Service  Schools  Conmand  in  Great  Lakes,  Illinois  to 
enhance  the  training  capabilities  of  the  basic 
machinist  mate  and  boiler  technician  schools.  A 
second  “Hot  Plant"  was  to  be  constructed  at  the 
Surface  Warfare  Officers  School  Command  in  Newport, 
Rhode  Island  to  support  that  school’s  officer  and 
enlisted  engineering  courses.  However,  for 
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economic  reasons  it  was  decided  that  the  second 
training  plant  should  be  a  simulator  1200  PSI 
Propulsion  Plant  Trainer.  There  was  considerable 
opposition  to  the  simulator  trainer  approach 
versus  a  "Hot  Plant",  but  the  safety  and  economic 
factors,  plus  manpower  savings,  were  overwhelmingly 
in  favor  of  the  simulator. 

DEVICE  19E22  TRAINING  OBJECTIVES 

The  1200  PSI  Propulsion  Plant  Trainer  student 
training  programs  designed  to  meet  the  trainer's 
mission,  which  is  to  provide  the  student  with  the 
prerequisite  knowledge  and  skills  necessary  to 
competently  commence  conventional  steam  propulsion 
plant  qualifications,  are  structured  to  meet  the 
following  five  training  objectives: 

1.  Develop  the  ability  to  properly  start, 
operate,  and  secure  the  plant  under  normal 
conditions. 

2.  Develop  the  ability  to  recognize  abnormal 
conditions  and/or  symptoms  which  signal 
an  impending  casualty. 

3.  Develop  the  ability  to  take  timely  and 
appropriate  corrective  action  which  would 
prevent  such  an  impending  casualty  from 
occurring. 

4.  Develop  the  ability  to  timely  place  the 
plant  in  a  safe  and  stabilized  condition 
when  a  casualty  does  occur  to  prevent 
cascading  effects. 

5.  Develop  the  ability  to  restore  the  plant 
to  normal  operating  conditions. 

Training  in  the  basic  skills  required  to  meet 
these  training  objectives  is  provided  by  the  vari¬ 
ous  officer  and  enlisted  personnel  service  schools. 
The  prime  mission  of  the  1200  PSI  trainer  is  then 
to  provide  the  necessary  "hands-on"  training  to 
shape  and  hone  those  skills  to  the  required  pro¬ 
ficiency  and  concerted  operation  levels.  Speci¬ 
fically,  Device  19E22  was  to  support  the  following 
training  courses: 

1.  Department  Head  Course 

•  Hands-on  training  prior  to  ship  billet 

2.  1200  PSI  Main  Propulsion  Assistant  (MPA) 
Course 

•  System  tracing 

•  E0SS  operational  indoctrination 

•  Casualty  control  procedures 

•  Maintenance  and  inspection 

•  Watch  standing 

•  Automatic  boiler  control  operation 

3.  1200  PSI  Supervisor's  Course 

•  Indoctrinate  junior  officers  and  SCPO's 
into  1200  PSI  operation  and  supervision 
of  maintenance 

To  meet  these  training  requirements,  a  simula¬ 
tor  training  device  based  on  the  engineering 
spaces  of  an  FF- 1078  class  frigate  was  decided 
upon  by  the  Navy. 


THE  TRAINER  AND  ITS  MISSION 

In  general  the  trainer  specification  required 
a  full  scale  reproduction  of  the  engineering 
spaces,  and  the  simulation  of  all  operational 
equipment  which  directly  support  the  generation  of 
electrical  power  and  the  propulsion  of  the  ship. 

In  addition,  the  equipment  simulation  should  in¬ 
clude  visual  and  aural  effects  associated  with 
both  normal  and  casualty  conditions. 

Figure  1  shows  the  configuration  of  the  trainer 
in  the  building  as  compared  to  the  relative  loca¬ 
tions  of  respective  engineering  spaces  aboard  the 
design  basis  ship.  As  ':ar  be  seen,  the  relative 
space  locations  are  i„e  Mme,  except  that  the 
Auxiliary  Machinery  Rot-"-  No.  2  and  Electrical 
Central  have  been  moved  in  order  to  accomodate  the 
trainer  within  a  more  reasonably  sized  building. 

The  basic  overall  trainer  dimensions  are  133' 
long,  41'  high,  and  47'  wide.  Although  direct 
maintenance  access  through  the  simulated  bulkheads 
is  provided,  they  are  hidden  from  direct  view  and 
are  locked  when  training  is  in  progress.  As  a 
result,  the  trainees  must  move  about  during  train¬ 
ing  exercises  in  the  same  manner  as  they  would  on 
board  ship. 

Operational  equipment  simulation  within  the 
engineering  spaces  uses  a  great  amount  of  detail 
to  ensure  easy  recognition,  and  to  facilitate 
familiarization  with  the  locations  of  the  various 
indicators  and  controls.  Figures  2  through  4 
illustrate  this  high  degree  of  realism  of  the 
simulated  equipments.  Of  note  is  the  considerable 
amount  of  piping  that  provides  system  tracing 
capability  to  the  trainer.  Essentially  all  of  the 
equipments  and  machinery  are  simulated.  In  a  few 
cases,  however,  real  shipboard  equipments  such  as 
alarm  panels  and  ship  convnuni cations  equipment 
are  used.  Equipment  simulation  includes  dynamic 
characteristics  such  as  start-up,  running,  shut¬ 
down,  casualty  modes  of  operation,  and  are  cate¬ 
gorized  into  the  following  sub-system  groups: 

•  Main  Steam  System 

•  1200  PSI  and  150  PSI  Desuperheated  Steam 
Systems 

•  Condensate  and  Feed  Systems 

•  Main  and  Auxiliary  Condenser  Air  Removal 
Systems 

•  Main  Turbine  Lube  Oil  Service  Systems 

•  Main  and  Auxiliary  Sea  Water  Circulating 
System 

•  Fresh  Water  Drain  System 

•  Fuel  Oil  System 

•  Auxiliary  Exhaust  and  Escape  Steam  Systems 

•  Lube  Oil  Systems 

•  Drain  Vent  and  Drain  Collecting  Systems 

e  Compressed  Air  System 

•  Automatic  Boiler  Control  Systems 

•  Electrical  Systems 

•  Communications  Systems 

•  Twin  Agent  (AFFF  and  PKP)  Fire 
Extinguishing  System 

To  further  enhance  the  equipment  simulations, 
dynamic  aural  cues  are  extensively  used.  These 
aural  cues  are  electronically  produced,  and  are 
under  math  model  driven  computer  control  so  that 
full  operating  range  and  casualty  sounds  are 
realistically  generated.  Over  two  hundred 
speakers  and  four  sound  systems  are  used  which 
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1.  AUXILIARY  MACHINERY  ROOM  NO.  2 

2.  ENGINE  ROOM 
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7.  INSTRUCTOR  STATION  ANO  COMPUTER  ROOM 
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are  mounted  in  single  or  multiple  quantities  inside 
the  equipments,  thereby  approximating  very  closely 
the  sound  pattern  of  their  operational  ship 
counterparts. 

Visual  effects  are  also  used  in  many  instances 
in  order  to  produce  the  level  of  simulation  neces¬ 
sary  to  meet  the  training  objectives.  These 
effects  include  such  as  rotating  shafts  on  pumps, 
variable  water  levels  in  sight  glasses,  variable 
speed  and  direction  of  the  main  shaft,  burner 
flame  color,  stack  smoke  density,  and  conflagra¬ 
tion  in  the  Fire  Room.  As  in  the  case  of  the 
aural  cues,  the  visual  effects  are  also  under  math 
model  driven  computer  control  to  assure  their 
fidelity  to  the  operational  conditions  being 
simulated. 

All  electronic  equipment  necessary  to  control 
the  various  functionally  replicated  machinery  is 
located  within  the  major  machinery  units  in  each 
room  and  the  maintenance  accesses  to  these  are 
made  inconspicuous  so  as  not  to  detract  from  the 
realism  of  the  replication.  Cabling  too,  is 
either  hidden  from  view  or  forms  a  part  of  some 
simulated  ship's  piping  and/or  cabling.  In  gen¬ 
eral,  therefore,  the  internal  arrangement  of  the 
trainer  areas  resembles  very  closely  the  configu¬ 
ration  of  the  corresponding  areas  on  the  design 
basis  ship,  and  any  deviations  necessitated  by 
practical  considerations  are  Inconspicuous. 


This  attention  to  replication  detail,  compre¬ 
hensive  system  simulation,  and  the  extensive  use 
of  aural  and  visual  effects  creates  a  very  realis¬ 
tic  training  environment.  Consequently,  not  only 
can  a  broad  spectrum  of  training  in  normal  plant 
operation  and  casualty  controls  be  achieved,  but, 
in  addition,  such  training  is  conducted  under  the 
constraints  and  limitations  of  a  shipboard  en¬ 
vironment  without  the  dangers  inherent  in  that 
environment. 

As  important  as  a  realistic  training  environ¬ 
ment  is  to  meeting  the  training  objectives  of 
Device  19E22,  the  essence  of  this  trainer,  or  any 
trainer  as  a  matter  of  fact,  are  the  math  models 
which  drive  it.  Due  to  the  uniqueness  of  the  1200 
PS1  Propulsion  Plant  Trainer,  the  contractor 
exercized  considerable  freedom  in  the  design  and 
development  of  the  trainer,  so  as  to  take  full 
advantage  of  innovative  use  of  state-of-the-art 
simulation  techniques.  Consequently,  it  was  es¬ 
sentially  up  to  the  contractor  to  decide  upon  the 
best  math  modeling  approach  to  meet  the  training 
objectives. 

Basically  the  operational  requirements  were 
that  the  trainer  would  be  capable  of  supporting 
training  in  the  Engineering  Operational  Sequencing 
System  (EOSS ) ,  which  includes  Engineering  Opera¬ 
tional  Procedures  (EOP)  and  the  Engineering 
Operational  Casual ity  Control  (EOCC)  procedures. 
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Figure  2.  Fire  Room  Control  Station 


and  could  operate  in  the  following  four  modes  of 
operation: 

Integrated  Mode  -  Simultaneous  and  interactive 
operation  of  all  the  engineering  spaces  on 
common  training  problems. 

Independent  Mode  -  Separate  and  independent 
operation  of  three  rooms  -  Fire  Room, 
Engineer  Room  and  Auxiliary  Spaces  (con¬ 
sisting  of  Auxiliary  Machinery  Room  No.  1, 
Auxiliary  Machinery  Room  tio.  2  and 
Electrical  Central). 

Record/Playback  Mode  -  On  instructor  command 
record  a  30  minute  history  of  training 
data,  and  play  it  back  in  real  time. 

Accelerated  Mode  -  On  instructor  command  move 
from  one  training  situation  and  plant 
configuration  to  another  in  faster  than 
real  time. 

The  Record/Playback  Mode  is  used  for  critique 
and  demonstration  purposes,  and  the  Accelerated 
Mode  is  used  to  shorten  the  duration  of  operations 
such  as  lube  oil  preheating  cycles.  In  actuality 
the  Record/Playback  and  Accelerated  Modes  of 
operation  are  submodes,  since  they  are  active 
during  either  the  Integrated  or  Independent  Modes. 
Four  trainer  initialization  conditions  were  also 
specified.  They  were: 

Cold  Iron  -  Dockside  with  shore  steam  and 

shore  electrical  power. 


Auxiliary  Steaming  -  Dockside  on  one  boiler 
and  ships  electrical  power. 

Underway  I  -  One  boiler  in  automatic  control, 
15  knots. 

Underway  II  -  Two  boilers  in  automatic  control 
25  knots. 


It  should  be  noted  that  in  Integrated  Mode  the 
entire  trainer  will  be  set  to  the  selected  initial 
condition,  while  in  the  Independent  Mode  each  of 
the  three  rooms  (Fire  Room,  Engineer  Room,  and 
Auxiliary  Spaces)  can  be  set  to  any  of  the  tour 
initial  conditions.  Also,  although  a  large  num¬ 
ber  of  valves  and  controls  are  automatically  re¬ 
settable  on  computer  command,  some  manual  opera¬ 
tions  are  required  during  trainer  initialization. 
These  involve  such  operations  as  insertion  or  re¬ 
moval  of  burners,  rotary  switches,  and  the  like. 

To  provide  this  type  of  trainer  operation,  as 
well  as  support  comprehensive  training  in  plant 
operating  procedures,  it  was  decided  early  in  the 
design  development  stage  to  math  model  the  1200 
PSI  plant  from  the  physics  point  of  view.  By 
doing  so  the  math  models  would  recreate  in  essence 
the  physical  laws  that  gevern  the  plant's  opera¬ 
tion,  thus  assuring  accurate  simulation  of  the 
plant  dynamics  during  normal  operation  as  well  as 
casualty  effects  and  their  cascading.  Furthermore 


Figure  3.  Fire  Room  -  Firing  Aisle 


it  was  also  decided  not  to  model  the  exact  detailed 
physical  processes  that  take  place,  but  rather 
to  break  down  the  plant  into  the  smallest  necessary 
system,  subsystem,  component,  and  process  levels, 
and  then  use  lumped  parameter  math  modeling  tech¬ 
niques.  As  a  result,  while  the  math  models  still 
recreate  the  physical  laws  at  play  in  the  plant, 
they  are  simplified  considerably  and  have  the 
characteristics  of  a  black  box  with  inputs,  outputs, 
and  a  transfer  function  or  series  of  equations 
representing  the  physical  process.  Obviously, 
extreme  care  was  taken  during  this  breakdown 
process  to  assure  that  all  the  feedback  loops  were 
accounted  for  and  the  required  gage,  thermometer, 
indicator,  etc.  values  would  be  calculated. 

The  actual  math  modeling  process  used  a  bottom 
up  approach.  That  Is,  the  math  models  for  the 
lowest  level  were  developed  first,  then  for  the 
next  higher  level,  and  so  on  until  the  entire 
propulsion  plant  math  model  was  developed.  At 


each  math  modeling  stage  great  emphasis  was  placed 
on  making  sure  that  all  of  the  required  input/ 
output  variables  were  accounted  for,  and  that  the 
necessary  feedback  loops  were  incorporated.  One 
somewhat  unique  aspect  of  the  math  modeling 
process  was  that  electrical  equivalence  methods 
were  used  during  the  analysis  phase  to  gain  in- 
depth  unde; standing  of  the  plant's  operation, 
dynamics,  and  inter-relationships.  Thereafter  the 
models  were  written  using  the  appropriate  physics 
disciplines. 

There  were  156  casualties  called  out  in  the 
specification  which  could  be  inserted  and  removed 
on  instructor  conmand.  They  were  distributed 
among  the  three  trainer  areas  as  follows: 

Fire  Room  -  56 

Engineer  Room  -  29 

Auxiliary  Spaces  -  71 
156 
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Figure  4.  Auxiliary  Machinery  Room  No,  1  Ships  Service  Turbo  Generators  (SSTG's) 


Analysis  of  these  casualties  indicated  that  they 
fell  into  two  basic  categories.  They  were: 

1.  Equipment  or  component  failures,  such  as 
"Ahead  Throttle  Janwed", 

2.  Casualties  defined  by  symptoms,  such  as 
"Diesel  Fuel  Starvation". 

Prior  to  the  implementation  of  these  casualties 
into  the  math  models,  the  exact  mode  of  failure 
had  to  be  defined.  For  instance,  a  pump  failure 
can  be  due  to  a  broken  shaft,  overload,  burned- 
out  motor,  etc.  This  is  especially  true  in  the 
case  of  the  second  category  of  casualties  which 
are  only  defined  via  a  symptom.  For  example, 
there  are  a  relatively  larger  number  of  causes 
that  would  produce  fuel  starvation  in  a  diesel 
engine.  Furthermore,  many  such  effects  do  not 
occur  instantaneously,  but  rather  at  some  rate 
dependent  on  the  causing  mode  of  failure.  As  a 
result,  a  specific  failure  mode  must  be  defined, 
so  that  the  resultant  effects  will  be  accurate 
both  in  symptoms  and  the  time  frame.  Once  the 
exact  cause  or  failure  was  defined  for  each  of  the 
156  casualties,  the  plant  math  models  were  then 
appropriately  modified  to  react  realistically  to 
their  insertion  or  removal. 

Because  of  the  fact  that  the  plant  math  models 
were  written  based  on  the  physical  laws  of  the 


plant,  insertion  of  casualties  not  only  produced 
all  of  the  necessary  symptoms,  but  also,  if  appro¬ 
priate,  cascaded  even  to  catastrophic  propor<!ons 
if  timely  corrective  action  was  not  taken  by  the 
trainees.  Also,  although  all  of  these  casualties 
are  controllable  by  the  instructor,  a  number  of 
them  will  also  occur  as  a  consequence  of  inappro¬ 
priate  action  by  the  trainees.  As  an  example, 
turbine  rumble  will  result  if  incorrect  warm-up 
procedures  were  used.  In  some  instances  proper 
corrective  action  will  automatically  remedy  the 
cause  thereby  canceling  the  casualty.  In  the  case 
of  the  turbine  rumble  example,  the  appropriate 
reduction  in  turbine  speed  and  the  following  of 
proper  subsequent  heat-up  procedures  will  termin¬ 
ate  the  casualty  whether  it  was  initiated  by  the 
instructor  or  trainee  action.  In  many  cases, 
however,  the  failure  causing  the  casualty,  and 
therefore  the  casualty  also,  will  remain  in  effect 
even  if  proper  action  was  taken  by  the  trainees. 

A  case  in  point  is  a  failed  fuel  oil  pump  as  a 
result  of  a  broken  shaft.  The  corrective  action 
in  this  instance  is  to  place  the  standby  pump  on 
line.  However,  even  though  this  action  returns 
the  plant  to  normal  and  stable  operating  condi¬ 
tions,  the  failed  pump  will  remain  out  of  commis¬ 
sion  as  in  the  real  world  environment.  Thus 
training  in  degraded  plant  operation  capability  is 
also  provided,  since  unless  the  casualty  is  re¬ 
moved  by  the  instructor,  the  failed  pump  cannot  be 
returned  to  service. 
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To  provide  training  capability  in  both  the  cor¬ 
rect  normal  operation  and  casualty  control  proce¬ 
dures,  another  type  of  malfunction  was  added  to  the 
trainer.  These  were  called  raaloperations  or 
“MALOP" ,  and  are  flagged  and  identified  to  the  in¬ 
structor  whenever  the  proper  procedural  sequence  of 
operations  was  not  followed  by  the  trainees.  In 
those  cases  where  the  maloperation  affects  a  criti¬ 
cal  main  flow  system,  it  is  fully  Integrated  in  the 
math  models,  and  if  not  timely  corrected,  is  cap¬ 
able  of  producing  a  casualty.  Typically,  such 
maloperations  are  associated  with  the  fuel  oil, 
steam,  lube  oil,  or  sea  water  circulating  systems. 
On  the  other  hand,  if  the  maloperation  impacts  a 
noncritical  system,  it  will  only  be  sensed  by  the 
math  models  for  indication  to  the  instructor.  Mal¬ 
operations  of  this  nature  are  associated  with  vent, 
circulation,  drain,  gland  seal,  or  other  such 
valves.  In  general,  M4L0PS  will  remain  in  effect 
until  corrective  action  is  taken  by  the  trainees. 

In  those  cases  where  the  maloperation  affects  a 
critical  system,  however,  the  maloperation  pro¬ 
duces  a  degrading  effect  in  the  plant  math  models, 
and  if  corrective  action  is  not  taken  in  a  timely 
manner,  will  result  in  a  casualty. 

It  should  be  noted  that  the  purpose  of  the 
1200  PSI  Propulsion  Plant  Trainer  is  to  support 
comprehensive  training  in  proper  plant  operation 
under  normal  and  casualty  control  conditions  and 
not  to  support  training  in  plant  maintenance.  As 
a  consequence,  the  functionally  simulated  equip¬ 
ments,  components,  and  systems,  as  well  as  the  math 
models,  are  limited  to  those  relevant  to  plant 
operation.  Equipments,  components  and  piping  as¬ 
sociated  with  plant  maintenance  operations  are  only 
mocked-up  in  order  to  maintain  the  necessary  degree 
of  realism  and  to  provide  familiarization  to  the 
trainees.  The  decision  as  to  what  is  plant  opera¬ 
tion  related  and  what  falls  into  the  category  of 
maintenance  was  often  difficult  since  in  many 
cases  there  is  no  clear  cut  distinction  in  func¬ 
tion.  Thus  numerous  revisions  took  place  as  a 
result  of  re-analysis  conferences  with  Navy  per¬ 
sonnel  familiar  with  1200  PSI  plant  operations. 

The  drivers  of  the  Device  19E22  simulator  are 
the  computer  system  with  the  resident  math  models 
and  the  Real  Time  Interface  (RTI)  system  which  in¬ 
terconnects  the  approximately  5,000  trainer  input/ 
outputs  to  the  computer.  The  Device  19E22  com¬ 
puter  system  consists  of  a  Harris  Slash  4  computer 
system  with  120K  bytes  of  core  memory,  two  800  bpi 
tape  drives,  a  2150K  byte  fixed  head  disk,  and  the 
RTI  Adapter  (RTI A)  whose  purpose  is  to  make  the 
RTI  system  compatible  with  the  1/0  data  transfer 
formats  of  the  computer.  Of  interest  also  is  the 
configuration  of  the  simulated  Twin  Agent  Fire 
Extinguishing  System.  It  is  completely  independent 
of  the  main  computer  system  is  is  totally  micro¬ 
processor  controlled,  requiring  only  that  the 
trainer  room  lights  be  on  to  operate.  As  a  result, 
it  is,  in  effect,  a  trainer  within  a  trainer,  and 
permits  fire  control  training  in  conjunction  with, 
or  independent  of,  propulsion  plant  training. 

This,  in  conjunction  with  the  Independent  Mode 
capability,  enhances  the  trainer  up  time  through 
the  availability  of  several  modes  of  degraded 
operation. 

With  the  exception  of  the  Twin  Agent  Fire  Ex¬ 
tinguishing  System  trainer  which  is  self-starting 
and  self-contained,  total  control  of  trainer 


operation  is  provided  by  the  Instructor  Station 
shown  in  Figure  5.  It  is  divided  into  three  sec¬ 
tions,  as  follows,  from  left  to  right: 

1.  Fire  Room  Console 

2.  Engine  Room  Console 

3.  Auxiliary  Spaces  Console 

Each  console  consists  of  an  interactive  CRT/ 
Keyboard  terminal,  conmuni  cations  controls,  and  a 
special  purpose  control  panel.  The  interactive 
CRT/Keyboard  terminal  is  used  not  only  to  display 
pertinent  trainer  room  data,  but  also  to  initialize 
and  control  the  trainer  through  the  use  of  simple 
alphanumeric  data  entries.  The  only  discrete  con¬ 
trols  and  indicators  are  those  that  are  not 
readily  or  conveniently  adapted  to  CRT/Keyboard 
operation  and  are  located  on  the  special  purpose 
control  panel  to  the  right  of  the  CRT.  Typically, 
these  consists  of  Independent  Mode  controls  re¬ 
presenting  outside  of  the  room  system  demands 
and/or  supplies,  and  instruments  such  as  Engine 
Order  Indicator/Transmitter.  As  a  result,  in¬ 
structor  workload  is  greatly  minimized,  permitting 
him  to  devote  the  majority  of  his  time  to  train¬ 
ing  and  instruction. 

Communications  controls  for  each  instructor 
are  located  to  the  right  of  the  CRT.  These  con¬ 
trols  permit  the  instructors  access  to  all  of  the 
simulated  ships  conmuni cations  circuits  for  both 
monitoring  and  two-way  communications  purposes, 
as  deemed  necessary.  In  order  to  support  Integra¬ 
ted  and  Independent  Mode  trainer  operation,  the 
communications  system  is  under  computer  control. 
Thus  if  the  Integrated  Mode  is  selected,  normal 
inter-space  conmuni cations  are  possible,  with  the 
Instructor  Station  assuming  the  role  of  all  areas 
outside  those  simulated  in  the  trainer.  In  the 
Independent  Mode,  each  Instructor  Station  console 
takes  command  of  its  respective  counterpart.  That 
is,  the  Engine  Room  console  will  assume  the  role  of 
all  area*  outside  the  Engine  Room.  Additionally, 
a  wireless  instructor  communications  system  is  also 
provided.  This  system  enables  roving  instructors 
inside  the  trainer  to  communicate  with  the  Instruc¬ 
tor  Station  to  coordinate  training  exercises 
without  prewarning  the  trainees  of  upcoming 
situations. 

In  addition  to  the  critical  parameter  displays 
on  the  CRT's  and  the  monitoring  capability  of  area 
conmuni cations,  a  closed  circuit  TV  system  is  used 
to  further  enhance  the  instructor's  ability  to 
observe  and  evaluate  trainee  performance  during 
training  exercises.  This  system  has  nine  strate¬ 
gically  located  TV  cameras  inside  the  trainer  area, 
four  TV  monitors  on  the  Instructor  Station,  a  video 
switcher,  and  a  video  recorder.  Thus  not  only  can 
trainee  activity  be  monitored  during  a  training 
exercise,  but  also  a  recording  of  a  selected  exer¬ 
cise  segment  can  be  made  for  post-training  debrief¬ 
ing  and  critique  which  also  will  include  all  of  the 
communications  that  transpired  during  that  segment. 

This  then  was  a  general  description  of  Device 
19E22,  1200  PSI  Propulsion  Plant  Trainer,  its 
capabilities,  and  the  rationale  behind  the  hard¬ 
ware  and  software  design  approaches  that  were  used 
to  ensure  that  the  trainer  could  meet  the  require¬ 
ments  of  its  training  objectives,  and  thus  its 
mission.  Since  Device  19E22  represents  a  new 
approach  to  U.S.  Navy  engineering  training,  that 
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Figure  5.  19E22  Instructor  Station 


of  marrying  the  traditional  need  for  propulsion 
plant  and  power  generation  operator  training  with 
computer  technology  and  simulation  techniques,  it 
is  worthwhile  to  study  the  effectiveness  of  Device 
19E22  in  support  of  the  Navy's  training  efforts, 
specifically  in  support  of  the  training  efforts  of 
Surface  Warfare  Officers  School  Command. 

Surface  Warfare  Officers  School  Command  is  a 
shore  training  activity  operating  under  the  command 
of  the  Chief  of  Naval  Technical  Training 
(CNTECHTRA) .  The  mission  of  Surface  Warfare  Offi¬ 
cers  School  Command  is  to  provide  the  naval  surface 
warfare  forces,  through  a  system  of  functional 
training,  with  officers  professionally  qualified 
to  serve  as  effective  naval  leaders  of  surface 
ships  with  the  ultimate  goal  of  "command  at  sea." 
From  Ensign  to  Captain,  Surface  Warfare  Officers 
School  Command  provides  a  continuum  of  training 
for  surface  warfare  officers  at  every  level  within 
the  shipboard  organization  for  all  surface  ships 
in  the  fleet.  The  followina  Surface  Warfare 
Officers  School  Command  courses  emolov  Device 
19E22  directlv  in  their  curriculums.  Deoartment 
Head  Course.  Sol  it  Tour  Engineer  Officer  Course, 
and  1200  PSI  Main  Propulsion  Assistant  Course. 

Engineering  training  is  an  integral  part  of 
these  courses  since  surface  warfare  officers  are 
required  to  possess  comprehensive  knowledge  of 
surface  ship  propulsion  plants  and  electrical  power 
generation  and  distribution  systems.  Obtaining 
basic  engineering  qualification  is  a  prerequisite 
for  formal  qualification  and  designation  as  a 
Surface  Warfare  Officer.  Furthermore,  since  1976, 
qualification  as  a  shipboard  Engineering  Officer 
of  the  Watch  is  a  prerequisite  for  qualification 
to  command  at  sea.  The  existence  of  Device  19E22 


at  Surface  Warfare  Officers  School  Command  under¬ 
scores  the  importance  of  engineering  training  to 
the  career  path  of  today's  Surface  Warfare  Officer. 

As  aforementioned,  the  mission  of  the  1200  PSI 
Propulsion  Plant  Trainer  is  to  provide  the  student 
with  the  prerequisite  knowledge  and  skills  neces¬ 
sary  to  competently  commence  conventional  steam 
propulsion  plant  qualifications.  To  meet  this 
mission,  and  to  accomplish  the  training  objectives 
mentioned  earlier,  the  training  programs  of  Device 
19E22  and  the  engineering  curricula  of  Surface 
Warfare  Officers  School  Command  courses  are  struc¬ 
tured  around  the  FF-1052  Class  Frigate  Engineer¬ 
ing  Officer  of  the  Watch  (EOOW)  Personnel  Qualifi¬ 
cation  Standard  (PQS).  The  EOOW  PQS  system 
requires  the  student,  as  an  EOOW  trainee,  to 
possess  a  detailed  knowledge  of  the  theory  of  1200 
PSI  steam  propulsion  plant  operations  and  of  the 
many  main  and  auxiliary  systems  of  the  propulsion 
plant.  Engineering  theory  and  propulsion  plant 
systems  training  is  presented  in  classroom  lec¬ 
tures  at  Surface  Warfare  Officers  School  Conmand. 
Beyond  requiring  knowledge  of  theory  and  systems, 
the  EOOW  PQS  system  requires  the  trainee  to  dis¬ 
cuss  and/or  perform  certain  evolutions  at  the 
various  watchstations  in  the  propulsion  plant. 
Device  19E22  is  the  training  platform  on  which 
students  accomplish  PQS  watchstation  requirements 
during  normal  and  abnormal  plant  operating 
conditions. 

Matching  the  realism  of  Device  19E22  is  the 
fact  that  the  students  utilizing  the  device  are 
given  the  actual  EOOW  PQS  training  materials  that 
are  employed  on  the  operational  ships  themselves. 
Students  perform  watchstation  items,  such  as  plac¬ 
ing  into  operation  the  main  engine,  lighting  fires 
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in  a  boiler,  or  performing  corrective  action  for  a 
low  water  In  the  boiler  casualty,  and  earn  signa¬ 
tures  from  1200  PSI  Propulsion  Plant  Trainer  in¬ 
structors  In  the  EOOW  PQS  Qualification  Card.  These 
signatures  certify  that  the  student  has  demon¬ 
strated  to  a  qualified  operator  a  satisfactory 
ability  to  complete  a  specific  watchstation  action. 
The  qualification  card  serves  to  document  the 
actual  watchstation  training  the  student  received 
in  Device  19E22  and  is  forwarded  as  a  record  of 
training  to  the  Commanding  Officer  of  the  student's 
future  ship.  It  should  be  noted  as  a  practical 
matter  that  Surface  Warfare  Officers  School  Com¬ 
mand  does  not  attempt  to  "qualify"  students  as 
EOOW's,  for  that  formal,  at-sea  qualification  is 
the  prerogative  and  responsibility  of  the  surface 
ship's  Commanding  Officer.  Rather,  it  is  reported 
to  the  Commanding  Officer  what  qualification  pre¬ 
requisites  were  satisfactorily  completed  in  the 
1200  PSI  Propulsion  Plant  Trainer.  It  is  a  sig¬ 
nificant  fact,  however,  that  Commanding  Officers 
are  "accepting"  a  large  majority  of  the  EOOW  PQS 
line  items  performed  in  the  1200  PSI  Propulsion 
Plant  Trainer  as  valid  for  qualification  purposes 
aboard  their  ships.  Thus,  one  of  the  tangible 
benefits  of  the  1200  PSI  Propulsion  Plant  Trainer 
is  that  the  training  received  by  the  officer  in 
the  trainer  ashore  obviates,  to  a  significant 
degree,  the  need  to  receive  the  same  qualification 
training  at  sea.  The  end  result  is  that  officers 
are  receiving  their  formal  EOOW  qualification 
earlier  in  their  sea  tour  than  previously  exper¬ 
ienced,  and  can  dedicate  more  time  towards  meeting 
their  other  responsibilities.  The  readiness  of 
the  individual  officers  and,  thus,  the  readiness 
of  their  ships,  has  been  improved  because  of  the 
engineering  training  these  officers  receive  in 
Device  19E22. 

In  addition  to  using  the  actual  FF-1052  Class 
Frigate  EOOW  PQS  system,  the  students  also  utilize 
the  Engineering  Operational  Sequencing  System 
(EOSS)  operational  procedures  and  casualty  control 
procedures  that  are  identical  to  those  that  exist 
in  the  fleet  today  aboard  most  surface  ships. 
Disciplined  utilization  of  EOSS  is  stressed  at  all 
levels  of  the  trainer's  training  programs.  The 
1200  PSI  Propulsion  Plant  Trainer's  EOSS  is  a 
credible,  realistic,  and  workable  document.  It 
has  been  evaluated  as  satisfactory  and  technically 
correct  by  the  Naval  Sea  Systems  Connand.  It  was 
utilized  as  a  standard  during  acceptance  testing 
of  the  device,  and  It  was  taken  aboard  actual 
frigates  and  employed  to  safely  start  up  and  pre¬ 
pare  the  propulsion  plant  for  underway  operations. 

Device  19E22  EOSS  and  PQS  programs  are  the 
keystones  of  its  student  training  programs.  These 
documents  are  no  different  than  their  shipboard 
counterparts  and  match  the  realism  of  Device  19E22 
itself.  Utilization  of  EOSS  and  PQS  enhance  the 
capability  of  the  trainer  to  provide  effective 
training.  The  capability  to  provide  training  and 
practice  by  students  in  the  disciplined  utiliza¬ 
tion  of  the  EOSS  while  actually  performing  Engi¬ 
neering  Officer  of  the  Watch  qualification  pre¬ 
requisites  is  an  important  element  of  the  engineer¬ 
ing  training  offered  at  Surface  Warfare  Officers 
School  Connand  and  one  which  was  not  available 
until  Device  19E22  was  placed  into  operation. 

Device  19E22  was  commissioned  In  November  1977 
and,  after  a  series  of  acceptance  tests  and  reli¬ 
ability  and  maintainability  demonstrations,  was 


placed  into  operation  for  student  training  in  late 
February  1978.  Through  30  June  1980,  the  trainer 
has  been  utilized  to  provide  training  to  nearly 
3000  students,  totaling  over  100,000  nan-hours  of 
training.  During  this  same  period,  the  reliability 
of  the  device  itself  has  been  exceptionally  high. 

In  28  months  of  continual  daily  operation,  only 
89  of  over  6100  scheduled  training  hours  have  been 
lost  due  to  unscheduled  maintenance  or  repair 
activity.  Both  Surface  Warfare  Officers  School 
Command,  as  the  custodian  of  this  one-of-a-kind 
training  device,  and  the  Navy  have  been  exception¬ 
ally  pleased  with  and  proud  of  the  capabilities 
of  Device  19E22,  the  reliability  of  the  trainer, 
the  efficiency  of  training  conducted  on  the 
device,  and  the  training  effectiveness. 

When  Device  19E22  was  placed  into  operation 
for  student  training,  a  pilot  program  was  estab¬ 
lished  for  a  1200  PSI  Main  Propulsion  Assistant 
Course  class.  Initial  utilization  of  the  device 
was  directed  for  this  course  because  the  students 
of  this  course  receive  no  other  functional, 
"hands-on,"  deckplate  training,  enroute  to  their 
ultimate  duty  station,  other  than  that  available 
in  the  1200  PSI  Propulsion  Plant  Trainer.  The 
pilot  program  was  expectedly  successful.  The 
capabilities  of  the  device,  including  the  realis¬ 
tic  simulation,  the  magnitude  of  casualties,  and 
reset  capability,  all  combined  to  provide  a 
dynamic,  five-week  addition  to  the  1200  PSI  Main 
Propulsion  Assistant  Course.  It  was  clear  that 
more  operational  training  could  be  accomplished 
in  Device  19E22  than  in  a  shipboard  style,  inte¬ 
grated,  hot  plant. 

The  positive  results  of  Surface  Warfare 
Officers  School  Conmand's  training  program  in 
Device  19E22  for  the  1200  PSI  Main  Propulsion 
Assistant  Course  led  to  a  rapid  integration  of 
trainer  utilization  into  the  Department  Head 
Course  curriculum.  Other  factors  contributed  to 
utilization  of  the  trainer  by  this  course  com¬ 
mencing  in  May  1978.  Heretofore  "hands-on", 
engineering  training  for  students  enrolled  in  this 
course  was  accomplished  at  sea  aboard  various 
units  of  the  Naval  Surface  Force,  U.S.  Atlantic 
Fleet.  The  reduction  of  surface  units  in  the 
Atlantic  Fleet,  the  increased  operating  tempo  of 
the  remaining  available  SURFLANT  ships,  the  rising 
fuel  costs  for  operating  ships  at  sea,  the  econ¬ 
omic  costs  of  per  diem  and  travel  to  and  from  the 
homeports  of  the  training  ships  for  the  Depart¬ 
ment  Head  Course  students,  and  the  uncertain 
material  condition  of  the  training  ships'  propul¬ 
sion  plants  which  on  some  occasions  caused  loss  of 
training,  accelerated  the  integration  of  the 
Department  Head  course  into  the  utilization  sche¬ 
dule  of  Device  19E22. 

With  the  addition  of  the  Department  Head 
Course,  which  placed  an  annual  demand  on  the 
trainer  of  12  training  weeks,  to  the  1200  PSI 
Main  Propulsion  Assistant  Course,  which  required 
access  to  the  trainer  for  50  weeks  annually, 
scheduling  conflicts  developed  which  required  con- 
tfnual  resolution.  Thus  a  decision  to  implement 
two  training  shift  operations  was  made  in  the  late 
summer  of  1978,  and  not  only  resolved  the  schedul¬ 
ing  conflicts  due  to  single  shift  operations,  but 
also  allowed  further  Increases  in  trainer  utiliza¬ 
tion,  thus  meeting  the  Increasing  demands  on  the 
trainer  by  new  training  Initiatives  of  Surface 
Warfare  Officer  School  Comnand.  In  the  interim 
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period  of  increasing  the  instructor  staff  to  sup¬ 
port  double  shift  operations,  the  Deputy  Chief  of 
Naval  Operations  (Surface  Warfare),  Vice  Admiral 
James  H.  Doyle,  Jr.,  USN,  established  the  follow¬ 
ing  1200  PSI  Propulsion  Plant  Trainer  Utilization 
Priority  Table  to  resolve  scheduling  conflicts 
between  current  and  future  users  of  the  trainer: 


Priority 

No. 

Course/Program 

Annual 

Requirements 

One 

1200  PSI  Main  Propul¬ 
sion  Assistant  Course 
EOOW  Training  Program 

45  weeks 

Two 

Department  Head  Course 
(CORE)  Training 

8  weeks 

Three 

Department  Head  Course 
Engineering  Specialty, 
Split  Tour  Engineer 
Officer  Course 

4  weeks 

7  weeks 

Four 

Department  Head  Course 
Combat  Systems  EOOW 

12  weeks 

Five 

Fleet  Utilization 
Training  Program 

24  weeks 

100  weeks 

Single  shift  operations  continued  until  March 
1979  at  which  time  partial,  double  shift  training 
conmenced.  In  June  1979  two-shift  training  com¬ 
menced  on  a  full  time  basis. 

The  1200  PSI  Propulsion  Plant  Trainer,  because 
of  its  unique  capabilities,  provides  distinct  ad¬ 
vantages  to  the  students  of  the  various  courses 
and  training  programs  described  below.  Responses 
of  student  watchstanders  can  be  conditioned 
through  repetition.  The  reset  capability  allows 
training  evolutions  to  be  repeated  quickly  with 
negligible  loss  of  training  time.  Indeed,  in 
comparison  with  a  shipboard  propulsion  plant  in 
which  one  evolution  may  take  two  hours  to  accom¬ 
plish,  the  same  evolution  can  be  accomplished 
several  times  in  the  same  time  frame  in  Device 
19E22.  Because  of  the  many,  preprogramed  casual¬ 
ties  inherent  in  Device  19E22,  a  second  advantage 
is  that  the  students  are  permitted  to  observe, 
without  fear  of  the  consequences  of  non-action 
by  the  watchstanders,  the  actual  casualty  symptoms. 
Thus,  in  a  controlled  and  safe  environment  without 
time  constraints,  the  student  can  observe  the 
immediate  and  long  range  symptoms  and  effects  of, 
for  example,  a  loss  of  sea  water  circulating  water 
to  the  operating  condition  of  a  Ship's  Service 
Turbogenerator  and  to  the  electrical  distribution 
system.  Coupled  with  this  advantage  to  the  stud¬ 
ent  is  the  third  advantage  of  being  able  to  actually 
respond  to  the  casualty  symptoms  themselves,  to 
take  the  necessary  corrective  action,  in  accord¬ 
ance  with  Naval  Sea  System  Command  authorized 
Engineering  Operational  Casualty  Control  (EOCC) 
procedures,  to  bring  the  propulsion  plant  to  a 
safe,  stabilized  condition,  and  then  to  restore 
the  plant  to  its  original  operating  condition. 

Now,  Instead  of  merely  watching,  the  students  are 
doing  those  watchstander  actions  required  to 
properly  operate  and  control  the  propulsion  plant. 
Again,  as  described  earlier,  the  propulsion  plant, 
its  components  and  systems  will  respond  favorably 


or  unfavorably  to  student  watchstander  actions. 

Thus  the  trainer,  in  casualty  control  evolutions 
as  well  as  in  normal  operating  evolutions,  pro¬ 
vides  the  classic  teaching  environment  —  learning 
by  doing. 

A  fourth  advantage  to  the  students  made  pos¬ 
sible  by  the  design  of  Device  19E22  is  that  stud¬ 
ents  are  afforded  the  opportunity  to  perform  all 
propulsion  plant  evolutions  at  each  watchstation 
of  the  1200  PSI  Propulsion  Plant.  Students 
utilizing  the  trainer  are  actually  exposed  to  and 
receive  training  at  all  the  watchstations  that  are 
manned  in  the  FF-1078  propulsion  plant  during 
underway  Condition  One,  or  General  Quarters. 

Thus,  the  trainer  is  manned,  for  training  purposes, 
at  a  higher  level  than  an  operating  surface  ship 
would  be  for  normal,  routine  training  evolutions 
at  sea.  This  manning  level  permits  maximization 
of  training  for  the  student. 

The  fifth  advantage  to  the  student  is  closely 
linked  to  the  trainer's  capabilities  which  permit 
the  selection  of  training  scenarios  that  are  com¬ 
mensurate  with  the  abilities  of  the  student.  The 
spectrum  of  student  expertise  utilizing  the  1200 
PSI  Propulsion  Plant  Trainer  ranges  from  that 
possessed  by  a  fledgling  junior  enlisted  man 
stationed  aboard  ship,  to  the  detailed  knowledge 
and  experience  displayed  by  Lieutenant  Commanders, 
Limited  Duty  Officers,  and  Warrant  Officers.  The 
flexibility  of  Device  19E22  is  such  that  bene¬ 
ficial,  basic  individual  training  can  be  provided 
as  easily  as  sophisticated  team  training,  and, 
occasionally,  widely  disparate  types  of  training 
are  provided  simultaneously.  These  instances 
demonstrate  the  independent  and  concurrent  flex¬ 
ibility  of  the  trainer  inherent  in  its  design 
characteristics  and  capabilities. 

A  final  advantage,  and  perhaps  the  most  import¬ 
ant  one  in  view  of  its  long  term  effects,  is  that 
the  trainer  and  its  instructors  provide  the 
vehicle  by  which  the  students  are  schooled  in  the 
disciplined  utilization  of  Engineering  Operational 
Sequencing  System  (EOSS)  procedures  and  formal 
propulsion  plant  conmunications  doctrine  at  all 
watch  levels.  As  mentioned  earlier,  the  trainees 
learn  to  operate  the  propulsion  plant  and  all  its 
equipment  safely  in  accordance  with  formal, 
accurate,  authorized  documents.  They  are  indoc¬ 
trinated  into  respecting  these  procedures  and  in 
recognizing  the  need  for  strict  adherence  to  them. 
If  the  students  gain  nothing  from  their  experience 
in  the  1200  PSI  Propulsion  Plant  Trainer  other 
than  a  fervent  application  for  the  correct,  proper, 
and  right  way  in  which  to  operate  a  propulsion 
plant  in  strict  accordance  with  EOSS,  then  the 
trainer  has  still  provided  immeasurably  valuable 
training.  Students  are  conditioned  not  to  rely  on 
the  memory  of  the  watchstander  or  on  the  unsafe, 
unauthorized  "folklore"  approach  to  plant  opera¬ 
tions  that  regrettably  characterized  operating 
doctrine  in  the  past.  This  zealous  emphasis  on 
EOSS  is  more  a  function  of  the  1200  PSI  Propulsion 
Plant  Trainer  instructor  staff,  but  without  the 
device  as  the  practical  and  realistic  platform  on 
which  to  practice  utilization  of  EOSS,  then  the 
training  results  in  this  particular  area  would  be 
superficial  at  best. 

The  value  of  the  1200  PSI  Propulsion  Plant 
Trainer  is  reflected  in  its  heavy  utilization 
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schedule.  There  are,  in  fact,  more  training  de¬ 
mands  on  the  device  than  the  trainer  and  its  staff 
can  accommodate.  This  heavy  demand  underscores  the 
appreciation  of  the  1200  PSI  Propulsion  Plant 
Trainer  and  of  its  capabilities  by  Surface  Warfare 
Officers  School  Command  and  the  Navy.  Training 
requests  which  have  been  presented  to  Surface 
Warfare  Officers  School  Command  but  which  cannot 
be  regularly  and  formally  supported  by  the  trainer 
include  summer  training  for  U.S.  Naval  Academy 
midshipmen,  training  for  Naval  Reserve  Officer 
Training  Corps  (NROTC)  midshipmen  from  civilian 
universities,  and  U.S.  Naval  Reserve  Training. 
Additionally,  all  requests  for  Fleet  Utilization 
Program  Training  cannot  be  met  because  of  higher 
priority  commitments. 

The  Fleet  Utilization  Training  Program,  which 
places  a  yearly  demand  on  Device  19E22  of  24  weeks, 
represents  the  most  varied  utilization  of  the 
device,  and  is  the  one  program  which  can  be  ana¬ 
lyzed  in  terms  of  the  potential  for  expanded 
application  of  computer  simulation  of  propulsion 
plant  operations  for  the  greater  training  needs  of 
the  Navy.  Initial  second  shift  training  in  the 
1200  PSI  Propulsion  Plant  Trainer  was  conducted 
in  March  1979  in  support  of  this  program.  This 
program  evolved  from  a  proposal  by  Commander 
Destroyer  Squadron  Twenty-Eight  to  utilize  Device 
19E22  for  training  the  engineering  crews  of  the 
ships  in  that  squadron.  This  proposal  was  sanc¬ 
tioned  by  Vice  Admiral  James  H.  Doyle,  USN, 

Deputy  Chief  of  Naval  Operations  (Surface  Warfare) 
and  Vice  Admiral  William  Read,  USN,  Command er 
Naval  Surface  Force,  U.S.  Atlantic  Fleet, 
(COMNAVSURFLANT) .  A  pilot  program  was  established 
for  evaluation  during  March  through  June  1979, 
with  scheduling  the  responsibility  of 
COMNAVSURFLANT.  The  pilot  program  was  successful 
as  the  trainer  demonstrated  its  ability  to  pro¬ 
vide  safe,  energy- saving,  and  beneficial  training 
in  basic  steam  propulsion  plant  operations.  Since 
March  1979,  this  program  has  provided  valuable 
training,  with  tangible  results,  to  over  650  en¬ 
listed  personnel  of  16  different  Atlantic  Fleet 
surface  ships.  Several  of  these  ships  have 
utilized  the  1200  PSI  Propulsion  Plant  Trainer  on 
multiple  occasions.  Through  June  30,  1980 
nearly  14,000  man-hours  of  training  have  been  pro¬ 
vided  under  this  program.  These  ships  have  re¬ 
ported  that  the  differences  in  configuration  be¬ 
tween  their  propulsion  plants  and  that  of  the  1200 
PSI  propulsion  plant  trainer  have  had  negligible 
effect  on  the  benefits  and  value  of  the  training 
received.  Fleet  Utilization  of  the  trainer  has 
steadily  increased  to  its  present  level  of  24 
weeks  per  year.  This  program  provides  training 
opportunities  that  never  existed  before  and  now 
ensures  that  100%  utilization  of  the  device  is 
achieved.  When  scheduled  to  utilize  the  1200  PSI 
Propulsion  Plant  Trainer,  a  ship  preselects  a 
training  program  that  best  meets  its  particular 
desires,  based  upon  the  ship's  operating  cycle 
and  its  own  assessment  of  its  watchs tenders'  ex¬ 
perience  levels  and  training  needs.  The  five 
training  programs  available  to  fleet  users  are. 

1.  Basic  Knowledge:  Individual  training 
keyed  to  enlisted  watchstanders  without 
significant  steaming  experience. 


2.  Pre-LOE  (Light  Off  Examination):  Team 
training  for  watchteams  with  little  steaming  ship¬ 
board  experience. 

3.  Pre-LOE/Proficiency  Maintenance:  Team 
Training  for  match  teams  with  adequate  knowledge 
and  steaming  experience;  normal  start-up,  uperate 
and  secure  training. 

4.  Pre-OPPE  (Operational  Propulsion  Plant 
ExaminationJ/Proficiency  Maintenance  Team  Train¬ 
ing  with  sufficient,  integrated  steaming  exper¬ 
ience;  normal  operating  training  plus  some 
casualty  control  training. 

5.  Pre-OPPE/Pre-REFTRA  (Refresher  Training): 
Team  training  for  watch  teams  with  sufficient 
steaming  experience;  almost  exclusively  casualty 
control  training. 

The  flexibility  of  the  device  and  its  multiple 
modes  of  operation  easily  allow  a  ship's  crew  to 
transition  from  one  training  program  to  another  as 
it  gains  proficiency  in  operating  the  propulsion 
plant.  Scheduling,  as  mentioned  above,  is  the 
responsibility  of  COMNAVSURFLANT.  Of  interest  is 
that  when  notified  of  dates  the  trainer  is  avail¬ 
able  for  Fleet  Utilization  in  any  given  calendar 
quarter,  COMNAVSURFLANT  fills  the  schedule  within 
a  period  of  a  day  or  two.  The  fleet  units  are 
eager  to  use  Device  19E22.  Some  units  have  re¬ 
quested  several  months  in  advance  to  be  placed  on 
the  1200  PSI  Propulsion  Plant  Trainer's  Fleet 
Utilization  Schedule. 

Utilization  of  Surface  Warfare  Officers  School 
Command's  1200  PSI  Propulsion  Plant  Trainer  is 
maximized,  limited  by  necessity  for  maintenance 
and  daily  diagnostic  tests.  But  the  demands  for 
the  trainer  continue  to  grow,  particularly  from 
the  fleet.  The  track  record  of  Device  19E22  has 
been  most  impressive.  Its  utilization  as  an 
integral  part  of  several  Surface  Warfare  Officers 
School  Command  courses  has  added  dynamic  realism 
and  excitment  to  these  courses.  It  has  provided 
training  opportunities  where  none  existed  before. 
It  has  been  a  significant  cost-saver  as  students 
no  longer  travel  elsewhere  to  receive  ''hands-on" 
operator  training.  And,  although  obvious  in  an 
era  of  skyrocketing  energy  costs,  it  is  noted  that 
the  trainer  does  not  consume  fuel  oil  or  lube  oil. 
Device  19E22  provides  functional  engineering 
training  at  a  cost  substantially  lower  than  any¬ 
where  else  in  the  Department  of  Navy.  The  1200 
PSI  Propulsion  Plant  Trainer  freed  Atlantic  Fleet 
units  from  training  commitments  to  Surface  Warfare 
Officers  School  Comnand's  Department  Head  Course, 
thus  allowing  them  to  dedicate  their  fuel  alloca¬ 
tions  for  underway  activities  more  closely  aligned 
to  their  own  primary  mission  areas  and  commit¬ 
ments.  Qualification  prerequisites  performed  in 
the  1200  PSI  Propulsion  Plant  Trainer,  and  ac¬ 
cepted  by  students'  Commanding  Officers,  have 
reduced  qualification  periods  of  the  students 
aboard  ship. 

The  1200  PSI  Propulsion  Plant  Trainer  has 
proven  Itself  exceedingly  safe  and  reliable  and 
highly  capable  of  providing  efficient  and  effec¬ 
tive  training.  Computer  technology  and  simulation 
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techniques,  exemplified  in  Device  19E22,  are  indeed 
a  present  and  viable  means  to  meet  the  ever- 
increasing  training  needs  of  the  U.S.  Navy,  par¬ 
ticularly  in  the  surface  engineering  conmunity. 
Surface  Warfare  Officers  School  Comnand  could  not 
be  more  pleased  with  the  positive  impact  Device 
19E22  has  had  on  its  training  programs.  This  en¬ 
thusiasm  for  the  present  performance  and  potential 
of  the  1200  PSI  Propulsion  Plant  Trainer  has  been 
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ABSTRACT 

The  training  requirements  for  the  nuclear  submarine  engineering  watchkeeper  are 
stringent.  The  Royal  Navy  uses  simulators  for  initial  and  continuation  training.  A  typical 
submarine  propulsion  system  is  described.  A  typical  simulator  is  also  described  with 
reference  to  the  hardware  provided,  the  degree  of  simulation  and  the  training  facilities 
provided.  Training  on  the  simulator  is  compared  with  training  on  actual  plant,  and  the 
relative  advantages  and  disadvantages  are  highlighted.  The  use  of  the  simulators  in  the 
overall  training  programme  is  described,  commencing  with  ab  initio  training  where  the  use 
of  the  simulator  is  complementary  to  classroom  instruction,  through  qualification  and 
licensing  of  operators  where  operation  of  the  simulator  constitutes  part  of  the  qualifying 
examination,  to  continuation  training  where  the  simulator  is  used  for  co-ordinated  team 
training.  The  programme  of  development  of  a  family  of  simulators  and  their  use  by  the  Royal 
Navy  is  described.  Some  of  the  lessons  learned  during  this  programme  are  discussed  with 
pointers  for  future  simulators. 


THE  TRAINING  REQUIREMENT 

It  is  common  for  nuclear  submarines  to  be  away 
from  base  for  many  weeks  at  a  time,  with  no  nuclear 
repair  facilities  at  ports  of  call.  Consequently, 
the  burden  of  support  is  placed  on  the  crew.  As 
crewmen  must,  therefore,  be  totally  familiar  with 
both  the  operational  and  the  maintenance  aspects  of 
the  plant,  intensive  training  is  directed  towards 
this  total  knowledge.  Safety,  correctness  and 
confidence  are  instilled  as  essentials  throughout 
the  stringent  training  of  these  nuclear  submariners. 

Training  in  maintenance  and  operation,  whilst 
complementary,  require  different  equipment. 
Maintenance  training  usually  requires  practice  on 
actual  equipment;  however,  if  the  prime  equipment 
is  used,  it  is  probable  that  the  full  range  of 
operations  could  not  be  readily  presented  without 
perhaps  endangering  the  plant  or  the  personnel. 

Practice  in  real-time  operations  (without  the 
expense,  hazards  or  restrictions  of  a  real  plant) 
is  therefore  most  readily  afforded  by  the  use  of 
simulators.  The  Royal  Navy  uses  simulators  for 
both  the  ship's  control  room  and  the  engineering 
plant  control  room  (Machinery  Control  Room)  the 
latter  being  the  subject  of  this  paper.  Practice 
Is  essential;  initially  to  demonstrate  control  of 
the  plant  within  its  normal  operational  boundaries 
and  subsequently  to  provide  experience  of  various 
degrees  of  emergency  operations  resulting  from 
incidents  such  as  plant  failures  and  maloperations 
by  other  crew  members.  Used  effectively,  simulator 
training  will  engender  confidence  through  practice 


and  familiarisation  of  all  situations,  including 
those  which  are  expected  to  be  rarely  encountered 
on  the  operational  vessel . 


THE  BOAT  PROPULSION  SYSTEM 

A  typical  boat  propulsion  system  is 
illustrated  in  Figure  1. 

The  heart  of  the  power  plant  is  the 
pressurised  water  reactor.  This  is  a  collection 
of  fissile  uranium  fuel  elements  arranged  in  a 
shielded  pressurised  vessel.  The  fission  rate  in 
the  reactor  is  controlled  by  use  of  special  rods 
which  are  inserted  into  the  fuel  to  absorb  fission 
neutrons.  The  reactor,  steam  generator, 
pressuriser,  coolant  pump  and  pipework  form  an 
enclosed  loop  referred  to  as  the  primary  circuit. 
Water  coolant  is  pumped  round  the  circuit,  passing 
through  the  reactor  where  it  extracts  heat  from 
the  nuclear  fuel  elements. 

The  purpose  of  the  pressuriser  is  to  maintain 
the  coolant  water  at  a  steady  state. 

Electrical  heaters,  mounted  Inside  the 
pressuriser  vessel,  are  used  to  maintain  a  steam 
bubble  within  the  pressuriser  vessel  thereby 
maintaining  the  pressure  of  the  coolant  system  at 
the  correct  saturation  pressure  to  achieve  this 
steady  state. 

After  passing  through  the  reactor  the  heat 
coolant  circulates  through  the  tubes  of  the  steam 
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generator,  where  its  heat  is  used  to  convert  the 
low  pressure  water  outside  the  tubes  into  high 
quality  steam.  The  now  cooler  primary  water  is 
circulated  through  the  reactor  to  continue  the 
heat  exchange  cycle.  The  complete  primary 
circuit  is  housed  in  a  specially  shielded  reactor 
compartment  to  contain  harmful  radiations. 

Steam  from  the  secondary  side  of  the  steam 
generator  is  used  to  drive  the  main  propulsion 
turbines  which,  through  reduction  gears,  drive 
the  propeller  shaft.  On  leaving  the  main  turbines 
the  spent  steam  is  condensed  in  a  sea  water 
cooler  and  returned  to  the  steam  generator  as 
water  to  begin  the  secondary  cycle  once  again. 

Steam  is  also  used  to  drive  turbo-generators 
which  provide  AC  electrical  power.  DC  power 
requirements  are  met  by  motor  generators  or 
batteries.  Spent  steam  from  the  turbo-generators 
is  recycled  in  the  same  way  as  that  from  the  main 
turbines. 

Sophisticated  automatic  and  manual  controls 
provide  the  controlling  means  for  both  the  reactor 
and  steam  generator  facilities  and  also  for  the 
secondary  and  support  facilities.  In  addition 
elaborate  electronic  and  mechanical  features 
assure  the  safety  of  the  nuclear  reactor  at  all 
times.  These  controls  are  mounted  on  a  console 
structure  situated  in  the  Control  Room.  It  is  this 
control  complex  which  is  simulated. 


THE  SIMULATOR 

A  typical  Machinery  Control  Room  Training 
Simulator  complex  is  illustrated  in  Figure  2. 

The  complete  complex  comprises  the  following  three 
major  subsystems 

1.  Machinery  Control  Room. 

2.  Instructor's  Console. 

3.  Computer  and  Interfaces. 

The  first  two  of  these  subsystems  comprise 
the  instructional  complex;  the  latter,  which  is 
situated  in  a  separate  room,  the  simulator  control 
complex. 

The  Machinery  Control  Room  is  an  exact 
replica  of  the  Ship's  Machinery  Control  Room  with 
the  hull  skin  removed.  The  lower  area  between 
the  ribs  is  open  thereby  affording  the  Instructor 
the  means  to  observe  directly  the  operation  of  the 
trainee  crew.  All  controls,  instrumentation  and 
equipments  normally  visible  to  and  operated  by  the 
crew  are  fitted.  The  appearance,  location  and 
operation  of  these  correspond  to  those  which  exist 
in  the  actual  submarine.  Two  free-standing 
loudspeaker  units,  for  the  reproduction  of 
simulated  noise  effects,  are  situated  one  at  each 
end  of  the  main  console,  though  not  directly 
visible  to  the  trainees. 

The  Instructor's  station  comprises  a  console 
and  a  raised  platform  which  enables  the  Instructor 
to  overlook  the  interior  of  the  Machinery  Control 
Room  and  observe  trainee  action.  In  addition, 
there  Is  an  observation  platform  whereby  personnel 
not  directly  Involved  in  the  training  exercise  can 
observe  the  proceedings.  The  console  contains  all 


necessary  instrumentation  and  controls  which 
enable  the  Instructor  to  initialise  the  training 
exercise,  to  monitor  trainee  operation,  where 
necessary  to  act  as  an  out  station  operator,  to 
provide  override  of  the  Machinery  Control  Room 
controls  and  to  induce  faults  in  the  various 
simulated  submarine  systems. 

Situated  in  a  separate  room  is  the  computer/ 
interface  complex  which  comprises  the  following 
major  i terns :- 

a.  A  general  pupose  digital  computer. 

b.  An  interface  cabinet. 

c.  A  junction  box. 

The  general  purpose  digital  computer  comprises 
central  processor,  core  store  memory  modules, 
magnetic  disc  backing  store  and  control  station 
for  systems'  initialisation  and  software 
maintenance.  System  modelling  is  by  core 
resident  software  programs. 

The  interface  cabinet  contains  the  circuitry 
which  connects  the  Machinery  Control  Room  controls 
and  display  instrumentation  (analogue  and  digital 
signals)  to  the  computer.  It  also  contains  the 
circuit  necessary  to  synthesize  the  audio  effects, 
the  production  of  which  is  under  software  control. 

The  junction  box  routes  signals  between 
the  interface  and  the  Machinery  Control  Room 
controls  and  displays. 

From  an  engineering  point  of  view.  Table  1 
summarises  some  of  the  significant  statistics 
of  the  simulator. 

Table  1 

SIMULATOR  STATISTICS 
Total  operational  software 


program  size  80k  words 

Typical  number  of  executed 
instructions  per  second  250k 

Number  of  digital  input 

signals  750 

Number  of  digital  output 

signals  600 

Number  of  analogue  input 

signals  65 

Number  of  analogue  output 

signals  150 

Total  number  of  Instructor 

input  faults  120 


The  following  two  sections  of  this  paper  give 
an  outline  description  of  the  simulator  with 
specific  references  to  the  two  major  operational 
complexes;  the  Machinery  Control  Room  and  the 
Instructor's  Console. 
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THE  MACHINERY  CONTROL  ROOM 

The  simulator  provides  full  simulation  of 
the  ship's  systems  as  described  below:- 

Primary  systems 

The  primary  systems  are  those  associated  with 
the  nuclear  reactor,  the  heat  source  of  the  steam 
propulsion  plant 

1 .  The  Reactor 

The  reactor  simulation  is  based  upon  a 
point  source  two  group  model,  which 
relates  neutron  power  and  primary 
coolant  temperatures  to  reactor  control 
rod  position.  The  model  used  takes  into 
consideration  all  relevant  effects  such 
as  reactor  core  age,  time  since  shut 
down,  previous  operating  power, 
operating  temperatures  and  xenon 
poisoning  levels.  In  addition,  the 
model  takes  into  account  rod  worth 
characteristics  and  instructor  induced 
fault  conditions  such  as  stuck  control 
rod,  single  or  group  rod  dropped  and 
instrumentation  channel  errors. 

2.  Reactor  Thermo  Dynamics 

This  model  simulates  the  transfer  of 
heat  from  the  reactor  fuel  plates 
through  the  pressurized  coolant  loops 
to  the  steam  generators;  the  effects  of 
the  main  coolant  pump  combinations  and 
the  state  of  main  isolating  valves  being 
taken  into  consideration. 

3.  Pressuri zer  System 

The  pressurizer  accommodates  variations 
in  primary  system  pressure  due  to  changes 
in  secondary  plant  power  demand. 

Because  of  the  extremely  high 
manoeuvrability  of  the  submarine,  these 
changes  can  be  both  large  and  rapidly 
occurring. 

The  pressurizer  system  model  simulates 
the  relationship  between  changes  in 
primary  circuit  temperatures  and  level 
changes  within  this  pressurizer.  In 
addition,  simulation  of  the  heater  and 
sprayline  control  logic  is  included 
together  with  their  dependence  upon  the 
electrical  system.  It  should  be  noted 
that  an  interaction  exists  between  some 
of  the  ship's  systems  (the  reactor, 
pressurizer,  steam  demand  and  power 
electrics)  which  are  all  simulated  in 
real  time.  Instructor  induced  faults 
can  also  be  inserted;  these  include 
indicated  parameter  errors,  control 
override,  fast  and  slow  primary  coolant 
leaks,  and  various  induced  alarms. 

4.  Associated  Systems 

Typical  such  systems  simulated  are  the 
emergency  cooling  system,  the  coolant 
make  up,  discharge  and  treatment 
systems,  the  coolant  pump  fresh  water 
cooling  systems. 


Secondary  systems 

The  secondary  systems  are  those  associated 
with  the  steam  propulsion  plant  and  include:- 

1.  Steam  generators  and  main  steam 
distribution  systems. 

2.  Feed  water  and  feed  water  regulation 
systems. 

3.  Main  engines. 

4.  Turbo  generators  -  for  main  electrical 
power  generation. 

Associated  auxiliary  systems,  typical  of 
which  are  the  circulating  water  systems,  the 
extraction  systems  and  the  lubricating  oil 
systems,  are  also  simulated. 


Electrical  systems 

These  comprise:- 

1. 

Battery. 

2. 

AC/DC  motor  generators. 

3. 

Turbo  generators. 

4. 

Shore  ac  electrical  supply 

All  circuit  breaker  inter-coupling  and 
protection  features  are  fully  simulated. 

Ship's  support  systems 

Air  conditioning,  hydraulics  and  other 
vital  systems  are  also  simulated. 

The  ship's  systems  form  an  integrated  closed 
loop  complex  in  which  a  change  of  state  in  one 
subsystem  is  reflected  through  other  subsystems. 
It  is  this  interaction  between  the  subsystems 
which  makes  not  only  the  design  of  the  simulator 
but  also  the  training  requirement  the  demanding 
tasks  that  they  are. 


THE  INSTRUCTOR'S  CONSOLE 

The  facilities  provided  at  the  Instructor's 
Console  allow  the  Instructor  to  perform  three 
main  functions.  These  functions  comprise  the 
following:- 

1.  Monitoring 

The  Instructor  has  a  view  of  the 
Machinery  Control  Room  through  the  cut 
out  in  the  roof  section  so  that  he  can 
monitor  the  actions  of  the  Operators. 
Meters  which  are  not  easily  visible  to 
the  Instructor  through  the  cut  out 
section  are  repeated  on  the  Instructor's 
Console.  These  meters  and  magnetic 
indicators  are  grouped  in  a  manner 
similar  to  those  on  the  Machinery 
Control  Room  panels. 

2.  Acting  as  an  'Out-Station'  Operator 

A  Machinery  Control  Room  operator  may 
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call  for  an  action  by  a  crew  member  who 
is  normally  located  elsewhere  in  the 
submarine.  To  cater  for  this,  and  to 
enable  the  Instructor  to  make  the 
normal  replies  to  the  operator,  'out- 
stations'  controls  are  provided. 

Typical  such  functions  are  associated 
with  the  main  machinery  space,  the 
electrical  systems  (such  as  running  up 
the  Diesel  Generators)  and  the  control 
room. 

3.  Defining  and  Controlling  Plant 
Conditions 

By  the  injection  of  plant  initial 
conditions  and  the  injection  of  system 
and  instrument  faults  the  Instructor 
can  define  and  change  plant  states 
prior  to  commencement  of  and  during  a 
training  exercise.  Thus  the  reactor 
initial  conditions,  comprising  core  age, 
power  level  of  previous  operation,  time 
since  shut  down,  desired  pressurizer 
and  heat  exchanger  levels  and  primary 
loop  temperatures  and  pressures  are 
specified  by  the  Instructor  as  system 
initial  conditions.  A  comprehensive 
range  of  faults  (a  total  of  120)  may  be 
injected  by  the  Instructor. 

Additional  realism  is  achieved  by  the 
inclusion  of  simulated  audio  effects,  the 
characteristics  and  onset  of  which  are  produced 
in  response  to  operator  actions.  Such  effects 
included  in  this  category  are:- 

a.  Audio  associated  with  the  running  up 
of  Diesel  Generators. 

b.  Circuit  breaker  operation. 

c.  Instructor  induced  steam  leaks. 

d.  Rod  drive  transmitter. 

Other  ambient  effects  such  as  temperature, 
humidity  and  vibration  are  not  reproduced. 

An  instructional  facility  provided  with 
the  simulator  is  that  of  record/ replay.  This 
facility  enables  the  Instructor  to  select  the 
whole  or  any  part  of  an  exercise  for  recording 
and  subsequent  replay  for  debriefing  purposes. 

The  display  parameters  on  all  meters  and  the 
state  of  all  indications  are  recorded  at  fixed 
intervals.  These  recordings  can  be  replayed  at 
different  rates  specified  by  the  Instructor 
(faster  or  slower  than  real-time)  on  to  the 
Machinery  Control  Room  instrumentation.  The 
Instructor  carv  start  the  replay  at  any  point  in 
the  recording.  Using  a  faster  than  real-time 
replay  long  term  transient  effects  can  be  readily 
demonstrated.  Using  the  slower  than  real-time 
replay  high  speed  effects,  such  as  the 
relationship  between  circuit  breaker  operation 
and  electrical  generator  states,  can  be 
demonstrated. 


THE  ADVANTAGES  OF  SIMULATOR  TRAINING 

Most  of  the  advantages  of  using  a  simulator 
for  training  purposes  can  be  summed  up  by  saying 


that  it  is  relatively  inexpensive,  much  safer 
and  allows  training  for  emergencies  which  cannot 
be  conducted  on  a  real  plant.  It  is 
significantly  safer  to  train  the  watchkeepers  on 
the  simulator  than  to  train  them  on  the  prime 
equipment.  The  savings  in  cost  are  the 
consequence  of  many  features,  but  predominant 
among  them  are  the  low  capital  and  running  costs 
of  a  simulator  by  comparison  with  the  equivalent 
figures  for  a  real  set  of  machinery.  Not  only 
are  the  operating  costs  minimal,  but  the 
simulator  can  be  operated  continuously  and 
exactly  repeatably  for  year  after  year,  whereas 
the  real  plant  must  necessarily  be  withdrawn 
from  service  for  refitting  at  regular  intervals. 
Furthermore,  variants  of  one  simulator 
representing  several  submarines  makes  for  even 
greater  savings  in  cost. 

Further  cost  savings  result  from  the 
ability  to  locate  a  simulator  at  any  convenient 
place,  such  as  within  a  training  establishment, 
or  even  to  make  it  transportable.  In  addition, 
the  simulator  can  be  brought  to  any  desired 
plant  state  in  a  few  seconds,  whereas  a  major 
change  of  state  may  require  several  hours  with 
real  machinery. 

The  ability  to  train  for  hazardous  or 
emergency  conditions,  which  is  another  major 
advantage  of  simulator  training,  is  out  of  the 
question  when  training  with  a  real  plant.  Since 
it  is  of  crucial  importance  that  the  watchkeepers 
can  recognise  and  respond  correctly  to  emergency 
situations,  which  by  their  nature  are  encountered 
only  very  rarely  in  real  life,  the  value  of 
simulator  training  in  this  context  can  hardly  be 
overstated.  Not  only  can  such  exercises  be 
carried  out  they  can  also  be  repeated  as  often 
as  desired. 

It  is  worthwhile  at  this  point  to  highlight 
two  features  commonly  found  in  all  good 
simulators.  One  is  the  facility  to  change  the 
pace  of  the  action  to  permit  recognition  of  the 
constituent  stages,  without  diminishing  the 
logical  validity;  in  the  limit  this  may  include 
single  stepping,  or  stills,  or  even  freezing  the 
simulation  state  at  any  point  to  allow 
discussion.  The  other  feature  is  the  accurate 
repeatability  of  the  simulation  when  generated  by 
a  computer  program.  It  is  >  ighly  unlikely  that 
a  plant  transient  or  incident  can  be  accurately 
repeated  on  demand  on  a  real  plant,  but  because 
the  'plant'  of  a  simulator  is  a  programmed 
computer  the  repeatability  is  total. 

A  particular  advantage  is  that  operator 
training  can  be  started  before  the  real  plant 
is  available  which,  apart  from  savings  in  time, 
certainly  makes  the  initial  operation  of  a  new 
type  of  plant  safer  and  easier  and  provides  a 
pool  of  operators  as  part  of  the  plant 
commissioning  team.  Adaptation  of  the 
mathematical  model  may  be  necessary  once  the 
operational  characteristics  of  the  plant  are 
available,  but  this  usually  is  more  fine  tuning 
than  significant  re-programming. 


Experience  has  shown  that  the  early 
availability  of  a  real-time  training  facility  is 
of  significant  advantage  and  has  also  shown  that 
the  optimum  lead  time  is  about  one  year.  This 
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provides  an  adequate  period  for  basic  operator 
training  for  a  new  plant;  if  a  longer  period  is 
programmed  then  the  computer  programs  and  the 
detailed  engineering  of  the  control  panels  may 
well  be  subjects  of  change  during  the  period  of 
simulator  construction.  In  practice  a  design 
freeze  at,  say,  a  one  third  stage  of  the 
simulator  build  programme  will  produce  a 
simulator  of  sufficient  accuracy  for  initial 
training  which  can  be  easily  modified  when  the 
true  characteristics  are  known.  A  simulator 
takes  of  the  order  of  two  years  to  construct, 
including  commissioning.  This  is  much 
shorter  than  the  time  required  to  build  and 
commission  a  nuclear  submarine  or  any  other 
plant  of  similar  complexity.  Consequently, 
the  simulator  build  can  be  started  at  a  point 
where  the  submarine  is  well  defined  and  still 
be  completed  in  time  to  provide  adequate 
training  prior  to  testing  the  real  power  plant. 

Thus  it  can  be  seen  that  genuine 
advantages  are  derived  from  the  use  of  a 
simulator,  some  of  which  can  be  fairly  readily 
quantified  in  terms  of  cost  or  timescale  with 
others  less  tangible. 


THE  DRAWBACKS  OF  SIMULATOR  TRAINING 

With  such  a  catalogue  of  benefits  to  be 
obtained  from  simulator  training  it  would  be 
surprising  if  there  were  not  some  accompanying 
disadvantages.  There  are,  of  course,  some 
problems.  The  chief  one  is  that,  no  matter  how 
perfect  the  simulator  may  be,  the  trainee  retains 
some  awareness  that  it  is  not  the  real  thing. 

His  motivation  to  do  well  is  thus  likely  to  be 
less  powerful  than  if  his  survival,  or  at  least 
his  comfort,  is  dependent  on  his  performance. 

This  is  a  psychological  effect  and  no 
doubt  varies  in  its  significance  between 
individuals,  but  it  is  always  present  to 
some  degree.  Not  only  can  his  motivation 
be  affected  by  this  factor,  but  there  can 
perhaps  be  an  attitude  of  recklessness 
induced  which  may  be  summed  up  as  "It's  only 
a  simulator,  so  let's  do  this  and  see  what 
happens'.  However,  this  attitude  may  not 
be  altogether  bad. 

The  second  difficulty  is  that  all  simulators 
have  less  than  complete  real  sm.  In  real  life, 
operators  obtain  their  total  information  input 
from  some  "unofficial"  sources,  as  well  as  the 
"official"  ones.  For  example,  the  fall  in  pitch 
of  the  hum  from  a  ventilating  fan  may  provide  a 
more  sensitive  indicator  of  falling  supply 
frequency  than  a  frequency  meter,  and  is  likely 
to  be  noticed  at  least  as  rapidly.  A  training 
simulator  is  unlikely  to  reproduce  all  the 
"ambience"  of  the  real  life  situation,  even  if  an 
attempt  is  made  to  do  so,  and  to  that  extent,  a 
simulator  is  unreal.  More  elaborate  simulation 
can  always  be  provided,  of  course,  but  only  at 
greater  cost  and  it  is  never  clear  at  what  degree 
of  realism  the  most  effective  training  is 
achieved,  especially  as  realism  is  somewhat 
subjective. 


THE  ROLE  OF  THE  SIMULATOR  IN 
OPERATOR  TRAINING 

Initial  training 

Simulators  may  be  used  through  all  the 
stages  of  training  and  refresher  training  of  the 
operators.  In  the  initial  stage  it  is  useful  to 
demonstrate  visually  the  concepts  taught  in  the 
classroom. 

It  is  essential  to  provide  practice  but 
also  desirable  to  eliminate  hazard  to  the  plant. 
Perhaps  this  can  be  compared  with  learning  to 
drive  an  automobile  where  basic  manoeuvres  or 
manipulations  produce  a  certain  fear  in  the 
Instructor,  especially  if  he  also  happens  to 
be  the  vehicle  owner.  Simulation  in  this 
stage  also  aids  the  appreciation  of  interactive 
systems  which  the  curriculum  inevitably  treats 
serially.  In  short,  topics  which  are  taught 
separately,  lesson  by  lesson,  can  be  presented 
simultaneously  if  that  is  how  they  occur  on  the 
real  plant.  Reference  has  been  made  earlier 
in  the  paper  to  the  total  repeatability 
provided  by  simulation,  and  use  of  this 
feature,  coupled  with  record/ replay  facilities, 
makes  an  essential  contribution  to  the 
transition  from  classroom  to  operational 
training. 

Thus  classroom  and  simulator  instruction 
can  be  complementary  in  the  initial  stages  of 
training. 

Qualification  and  licensing  of  operators 

The  next  stage  of  training  is  the  transfer 
from  the  general  application  of  the  basic  theory 
to  the  more  detailed  instruction  on  the  plant, 
or  submarine,  to  which  the  partially  qualified 
operator  is  drafted.  Any  one  simulator 
installation  can  be  fairly  readily  adapted  to 
accurately  represent  each  specific  vessel  of 
a  flotilla.  Although  in  some  instances  this  may 
necessitate  the  use  of  interchangeable  sections 
of  control  or  display  panels  it  may  also  only  be 
necessary  to  feed  into  the  computer  a  variant 
of  the  program  which  differs  parametrically  or 
logical ly. 

Perhaps  the  simplest  example  is  of  two  or 
more  similar  plants  with  slightly  different 
protection  systems  operating  at  different  trip 
values.  Even  if  panel  changes  are  involved 
this  can  usually  be  effected  speedily  (certainly 
compared  with  the  real  plant)  with  functions 
re-routed  by  the  computer  program  addresses. 

This  stage  of  specific  training  usually 
concludes  with  formal  qualification  (or 
licencing)  for  the  allocated  watchkeeping 
position,  where  a  period  of  intensive  operation 
in  the  simulator  constitutes  a  part  of  the 
qualifying  examination. 

The  logical  extension  of  this  stage  is  the 
training  and  formal  qualification  for  other 
positions  within  the  crew. 
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Continuation  training 

Training  is  not  only  a  phase  which  precedes 
qualification  and  practice.  It  continues 
throughout  the  operational  life  of  the  boat  to 
maintain  the  high  pitch  created,  to  introduce 
new  situations  presented  by  plant  modifications 
or  experiences  learned  across  the  fleet  and  to 
create  and  support  the  performance  as  a  team. 
Throughout  this  phase  emphasis  is  placed  on 
co-ordinated  team  training,  where  collective 
responsibility  and  the  interdependence  of  the 
various  watchkeeping  stations  can  be  demonstrated 
on  the  simulator  in  a  manner  which  it  would  be 
extremely  difficult  to  achieve  other  than  on  a 
plant. 

The  periodic,  or  refresher,  retraining  of 
teams  of  qualified  operators,  particularly  after 
submarine  remits,  and  the  requalification  after 
a  specified  period  of  time  may  also  utilise  the 
simulators  and  it  is  at  this  and  the  previous 
stage  that  intensive  emergency  operations  are 
practiced. 

Thus  the  simulators  play  a  vital  part  in 
every  stage  of  training  and  qualification  of 
operators  from  basic  principles  to  regular 
requalification.  Moreover  they  can  be  used 
effectively  in  the  assessment  of  operational 
drills,  the  ergonomic  effectiveness  of  new 
control  system  designs  and  the  validation  of 
operational  situations  postulated  in  safety 
assessments. 


THE  SIMULATOR  PROGRAMME 

The  Royal  Navy  has  been  using  simulators  for 
the  p'  rpose  of  training  Machinery  Control  Room 
watchkeepers  on  nuclear  submarines  for  more  than 
a  decade.  The  simulator  procurement  programme 
began  in  1966  with  the  placement  of  the  order  for 
the  Dounreay  based  simulator.  Prior  to  this 
date  training  had  been  undertaken  on  a  shore 
based  plant  which  fulfilled  the  dual  roles  of 
research  and  development  and  training.  The  first 
refit  of  this  prototype  plant  necessitated  the 
introduction  of  a  simulator  to  take  over  the 
training  aspects.  At  this  time  the  full 
potential  of  simulation  techniques  was  not 
recognised  and  as  a  consequence  a  general  purpose 
simulator  was  designed;  it  was  thus  not  fully 
representative  of  any  particular  ship  type. 
Notwithstanding  this  its  operational  usage  more 
than  proved  the  case  for  sucn  training  simulators 
and  consequently  contracts  were  placed  for  the 
second  and  subsequent  simulators. 

There  is  now  a  total-  of  six  Machinery 
Control  Room  training  simulators  either  in 
service  or  under  build.  This  illustrates  a  long 
term  commitment  by  the  Royal  Navy  towards  the  use 
of  training  simulators.  There  is  every  indication 
that  this  commitment  will  continue  with  the 
procurement  of  additional  simulators  in  parallel 
with  predicted  fleet  expansion.  By  this  means 
the  Navy  will  maintain,  at  minimum  cost,  its 
customary  very  high  standard  of  operational 
readiness  of  its  watchkeeping  crews. 

A  great  deal  of  experience  has  been  accrued 
over  the  span  of  this  procurement,  development 
and  manufacturing  programme.  Some  aspects  of  this 


experience  will  be  outlined  from  the  point  of  view 
of  lessons  learned  and  consequent  pointers  towards 
improving  future  simulators.  It  is  not  claimed 
that  these  conclusions  in  any  way  represent  new 
discoveries  but  that  they  merely  confirm  existing 
good  practices  in  simulation  development  and 
procurement. 

From  the  manufacturer's  point  of  view 
experience  in  the  design  of  this  series  of 
simulators  has  re-emphasised  the  basic 
requirement  for  the  assembly  of  the  correctly 
qualified  design  team  and  the  production  of  a 
definitive  requirement  specification.  In 
this  respect  a  happy  marriage  between  the  skills 
of  Rolls  Royce  and  Associates  Limited  (RR&A) 
and  Marconi  Space  &  Defence  Systems 
Limited  (MSDS)  was  achieved.  Thus  RR&A  provided 
the  skills  of  the  actual  plant  designer,  the 
mathematician,  and  the  experienced  plant 
operator.  These  were  complemented  by  the  MSDS 
specialized  skills  in  respect  of  simulator 
design  and  in  particular  computer  hardware  and 
software. 

Bearing  in  mind  that  the  basic  requirement 
was  for  a  simulator  for  training  Navy 
watchkeeping  personnel  the  RR&A  experienced  plant 
operators  fulfilled  the  vital  role  as  the  expert 
co-ordinator  between  the  simulator  manufacturer 
and  the  eventual  user.  In  this  way  problems  of 
interpretation  and  of  a  subjective  nature  were 
in  most  cases  resolved  prior  to  involvement  of 
the  simulator  manufacturer. 

From  the  user's  point  of  view,  that  is  from 
the  Instructor's  point  of  view,  this  has  once 
again  confirmed  the  need  for  exceedingly  detailed 
and  careful  analysis  of  the  use  of  the  simulator 
as  a  training  tool  in  the  way  in  which  this 
affects  the  provision  and  layout  of  Instructor 
facilities.  Even  though  each  succeeding 
simulator  has  these  facilities  designed  into  it 
as  a  result  of  the  accumulated  experience  on 
preceeding  simulators  there  is  stll  room  for 
improvement  in  this  area.  This  leads  us  to 
consider  the  use  of  alternative  devices  for 
Instructor  control  and  display.  Whereas  existing 
simulators  have,  by  good  Navy  tradition,  such 
things  built  to  "battleship  standards",  a  more 
compact  and  cost  effective  and  operationally  more 
effective  facility  might  be  achieved  by  the  use 
of  visual  display  units  and  keyboards.  This  is 
an  area  which  might  bear  fruit  on  further 
consideration. 

Also  from  the  user's  point  of  view  more 
consideration  might  be  given  to  the  use  of  the 
simulator  for  purposes  other  than  for  training; 
for  example,  such  a  simulator  might  be  used  for 
the  trial  and  evaluation  of  new  operating 
procedures.  The  viability  of  this  is  obviously 
related  to  trainee  throughput  and  simulator 
availability  and  is  also  critically  dependent 
upon  the  sophistication  of  the  system  modelling. 
Consequently,  this  requirement  must  be  specified 
at  system  definition  phase  when  the  degree  is 
being  determined. 


SUMMARY 

From  the  trainee's  point  of  view, 
undoubtedly  a  very  high  standard  of  training 
is  being  achieved  with  the  simulators. 

However,  this  should  not  lead  to  complacency  - 
the  attitude  "what  we  have  works  well,  so  let's 
make  the  next  one  the  same"  must,  though 
seemingly  attractive  and  comforting,  be  resisted. 
Where  the  achievement  of  a  high  degree  of  realism 
in  any  simulator  is  expensive  and  obeys  the  law 
of  diminishing  returns  it  could  be  most 
profitable  to  investigate  a  methodology  whereby 
the  optimum  degree  of  realism  can  be  determined. 

In  these  circumstances  we  might  ask  such  questions 
as:  "Does  the  simulation  of  secondary  effects 
such  as  noise  significantly  enhance  the  training 
transfer?  Would  the  inclusion  of  additional 
ambient  effects  such  as  temperature,  vibration 
and  so  on,  significantly  increase  the  training 
transfer?"  Indeed  a  very  much  more  fundamental 
question  might  be  asked:  "Bearing  in  mind  that 
the  basic  requirement  is  the  training  in  a  set 
of  specific  skills,  is  the  best  training  device 


for  achieving  this  one  which  attempts  totally  to 
reproduce  the  boat  environment?" 
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ABSTRACT 


The  Submarine  Advanced  Reactive  Tactical  Training  System  (SMARTTS)  will  be  the 
“training  subsystem"  of  the  21A  Series  Submarine  Combat  System  Trainers.  SMARTTS  is 
the  result  of  a  series  of  research  and  development  investigations  directed  by  the  U.  S. 
Naval  Training  Equipment  Center  investigating  the  application  of  advanced  training 
technology  to  submarine  tactics  training.  SMARTTS  is  expected  to  greatly  enhance  the 
tactics  training  process  by  improving  the  quality  and  quantity  of  tactics  training  and 
by  correcting  current  deficiencies  of  the  tactics  training  system.  The  advanced 
training  technology  embodied  under  SMARTTS  will  present  emerging  concepts  and 
capabilities  that  should  be  initially  developed  as  an  Integral  part  of  every  simulator- 
based  training  system.  The  paper  summarizes  the  SMARTTS  program  placing  particular 
emphasis  on  the  development  of  the  SMARTTS  characteristics  as  a  result  of  a  modified 
instructional  systems  development  analysis.  The  SMARTTS  characteristics,  which  are 
primarily  implemented  via  software  additions  to  the  simulator  programs,  are  applicable 
to  virtually  every  simulator-based  training  system.  SMARTTS  will  be  a  major  addition 
to  the  Submarine  Combat  System  Trainers;  it  represents  a  milestone  in  that  it  is  a  major 
subsystem,  emphasizing  advanced  training  technology  as  an  integral  part  of  the  training 
device. 


INTRODUCTION 


The  Submarine  Advanced  Reactive  Tactical 
Training  System  (SMARTTS)  is  the  training  subsystem 
of  the  submarine  tactics  training  device.  SMARTTS 
has  been  specifically  designed  to  provide  training 
assistance  features  to  the  21A  series  Submarine 
Combat  Systems  Trainers  (SCSTs)  and  thereby  enhance 
their  training  effectiveness.  SMARTTS  is  designed 
to  address  Individual  and  team  training 
requirements  for  the  SSN  MK  117  fire  control  system 
and  AN/BQQ-5  sonar,  and  the  SSBN  MK  113  Mod  9  fire 
control  system  and  AN/BQQ-2  sonar.  SMARTTS  is  a 
strap-on  subsystem,  the  preprototype  of  which  is 
currently  being  installed  on  the  21A41  SCST, 
Norfolk,  Virginia.  An  evaluation  of  the  training 
effectiveness  of  the  21A41A  SCST  (i.e.,  with 
SMARTTS)  will  be  accomplished  in  the  operational 
training  setting  immediately  following  completion 
of  the  preprototype  installation. 

BACKGROUND  —  DEVELOPMENT  OF 

SUBMARINE  TRAINING  TECHNOLOGY 

The  U.S.  Navy  submarine  forces  are  experiencing 
accelerated  transition  to  significantly  Improved, 
although  increasingly  complex,  weapons  and  tactical 
comnand  and  control  systems  for  both  SSNs  and 
SSBNs.  These  technological  advances  are  matched  by 
expanding  submarine  mission  roles  and  tasks  in 
tactical  and  strategic  warfare,  and  other 
classified  operations.  Enhanced  operational 
performance  of  weapons,  fire  control,  sonar, 
periscopes,  electronic  support  measures,  and  other 
sensors,  together  with  advanced  ship 
characteristics,  places  heavy  emphasis  on 

Note:  The  authors  are  solely  responsible  for  this 
paper;  It  does  not  represent  U.  S.  government 
positions  or  policy. 


responsible  tactical  command  and  control.  High 
speed  digital  data  processing,  coupled  with 
computer  analysis  of  multiple-sensor  inputs 
requires  highly  developed  operator  visual  and 
interpretive  skills.  These  skills  must  be 
supported  by  the  operator's  ability  to  make  complex 
tactical  decisions.  The  number  of  personnel 
directly  involved  in  command  and  control  within  the 
submarine's  tactical  decisionmaking  processes  has 
likewise  increased  with  fleet  introduction  of  new 
technologically  advanced  systems.  These 
developments  are  placing  increasing  demands  on 
submarine  tactical  and  operational  training  to 
promote  operational  readiness. 

The  overall  submarine  tactics  training  system, 
whether  providing  formalized  team  training  in 
shore-based  attack  centers  or  individual  on-the-job 
training  at  sea,  principally  exists  to  support 
submarine  force  operational  readiness.  Submarine 
Combat  Systems  Trainers  (SCSTs)  for  SSNs  and  SSBNs 
are  located  at  all  major  submarine  operational 
training  centers.  Simulation  methodology  and 
equipment  retrofits  provide  the  latest  in  onboard 
equipment  developments  for  the  SCSTs. 
Collectively,  these  SCSTs  represent  the  primary 
component  of  the  submarine  tactical  training 
system. 

Tactics  training  presently  satisfies  a  wide 
range  of  requirements,  from  the  submarine  officer 
basic,  indoctrination,  and  advanced  courses 
(SOBC.SOIC,  and  SOAC)  through  advanced  levels  in 
the  prospective  executive  officer  (PXO)  and  the 
submarine  force  commander's  prospective  commanding 
officer  (PCO)  courses.  Training  within  these 
courses  spans  Individual  through  team  contexts. 
SCST  utilization  Includes  these  formal  shore-based 
courses,  as  well  as  support  for  ships  In  refresher 
and  predeployment  training.  Advanced  training 
technology  to  enhance  the  training  process  is 
required  for  these  SCSTs. 
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Until  recently,  with  the  advent  of  digitally 
processed  sonar  (e.g.,  AN/BQQ-5)  and  fire  control 
(i.e.,  MK  113  Mod  10  and  MK  117)  information  and 
advanced  weapons  (e.g,,  MK  48  torpedo.  Tomahawk), 
submarine  tactics  had  changed  little  since  World 
War  II.  This  was  generally  true  of  both  the 
hardware  systems  and  the  tactics  employed  in  their 
use.  Although  some  hardware  changes  did  evolve, 
they  were  primarily  the  result  of  better  hardware 
designs  to  perform  the  same  functions.  Several 
notable  exceptions  were  the  MK  51  analyzer  and 
several  plotting  techniques  (e.g.,  Eklund  ranging) 
which  were  developed  during  this  period. 
Nevertheless,  those  changes  that  did  occur 
represented  a  steady  evolution  in  submarine 
tactical  capability.  During  this  period,  the  Naval 
Training  Device  Center  (NTDC)  (which  is  currently 
the  Naval  Training  Equipment  Center  —  NTEC)  had 
initiated  a  variety  of  research  and  development 
efforts  investigating  the  improvement  of  training 
technology.  With  several  notable  exceptions,  the 
majority  of  these  efforts  were  concentrated  on 
surface  and  aviation  Navy  training  devices.  Much 
of  the  early  work  investigating  tactics  training  in 
the  submarine  force  was  performed  in  a  series  of 
NTDC/NTEC-studies  by  Sidorsky  and  others  during  the 

1960s. Sidorsky's  work  centered  on  tactical 
decisionmaking,  resulting  in  a  decisionmaking 
taxonomy  that  has  been  the  basis  for  several 
applications  of  decisionmaking  theory  in  the 
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operational  environment.  Hanmell  and  Mara1  '  used 
the  taxonomy  as  the  basis  of  an  approach  to 
submarine  tactical  decisionmaking  training. 
Observations  on  SCSTs,  discussions  with  naval 
training  and  operational  personnel,  and  analysis  of 
at-sea  exercise  data  indicated  the  potential  of 
this  approach  to  training.  A  later  effort  by 

Pesch,  Haimiell,  and  Ewalt/5^  used  the  Sidorsky 
taxonomy  as  one  level  of  a  multileveled  approach  to 
submarine  officer  tactical  decisionmaking 
training.  This  latter  approach  has  been  used 
successfully  to  train  merchant  ship  masters  in 
decisionmaking  regarding  the  international  rules- 

of-the-road.  ^ 

A  second  major  development  by  Sidorsky  was  the 
TACTRAIN  device,  an  individualized  CRT-based 
tactical  decisionmaking  trainer.  It  was  developed 
to  train  advanced  tactical  decisionmaking  to  senior 
submarine  officers.  TACTRAIN  introduced  several 
new  concepts  to  the  submarine  training  community. 
It  provided  individual ized  training  to  senior 
officers ;  it  was  based  on  a  CRT  driven  by  a  computer 
in  a  war-gaming  format;  it  enabled  a  wide  range  of 
flexibility  in  controlling  the  training  process; 
and  its  model  functioned  on  the  basis  of 
probability  estimates  (e.g.,  probability  of 
counterdetection).  At  that  time,  however,  the 
operational  submarine  fire  control  system  (e.g.,  MK 
113  Mod  6)  did  not  deal  with  probability  estimates 
or  display  such  data  on  CRTs. 

The  submarine  fire  control  systems  In  use  during 
the  mid-1960s  were  basically  similar  to  the  earlier 
fire  control  systems  in  function,  although  somewhat 
different  in  hardware  layout.  The  Submarine  School 
(SUBSCOL)  in  New  London  was  charged  with  training 
officers  entering  the  SOBC.  The  traditional 
approach  had  been  to  provide  individualized 
training  on  the  attack  director,  followed  by  team 
training  on  the  SCST.  A  group  of  individual 


torpedo  data  computers  (MK  4)  were  used  for 
individualized  training  with  regard  to  the  older 
basic  submarine  fire  control  systems  (MK  106). 
Individual  training  devices,  however,  were  not 
available  for  individualized  training  on  the  newer 
MK  75  attack  director  used  in  the  MK  113  fire 
control  system  series.  A  request  by  SUBSCOL  for  a 
group  of  individualized  MK  75  attack  director 
trainers  prompted  NTOC  to  initiate  an  investigative 
effort  to  determine  specific  submarine  officer 
requirements  for  tactics  trainers.  The  intent  was 
to  approach  the  tactical  training  issue  by 
systematically  analyzing  tactics  training  to 
determine  the  then  current  and  future  training 
device  requirements.  During  the  late  1960s,  a 
variety  of  fundamental  tactics  and  hardware  changes 
were  in  the  developmental  stage  (e.g.,  digital 
sonar  hardware,  digital  fire  control  hardware,  and 

48  torpedo).  These  advances  were  to  have  a 
profound  impact  on  submarine  tactics. 

The  investigation,  which  initially  considered 
basic  level  training  and  was  later  expanded  to 
include  advanced  officer  training  and  all  SCSTs, 
identified  a  variety  of  areas  in  which  the  tactics 

training  process  could  be  improved.^’ The 
identification  of  the  need  for  a  variety  of 
individualized  training  devices  together  with  the 
advanced  knowledge  of  the  computer/CRT-based  fire 
control  systems  under  development  at  that  time, 
resulted  in  the  recommendation  for  the  Generalized 
Individual  Trainer.  The  recommended  Generalized 
Individual  Trainer  was  to  be  a  general  purpose 
computer-based  CRT  console  that  could  be  readily 
configured  to  meet  the  wide  variety  of  training 
needs;  furthermore,  this  device  could  incorporate 
advanced  training  technology  concepts.  More 
importantly,  these  investigations  found  that  the 
current  SCSTs  lacked  fundamental  training 
technology  capabilities  which  had  the  potential  of 
greatly  enhancing  the  training  process.  The 
authors  concluded  that  if  these  capabilities  were 
incorporated  into  the  training  device  when 
initially  developed,  they  would  represent  a 
relatively  small  increase  in  cost.  A  variety  of 
training-related  capabilities  were  recommended  to 
be  installed  on  the  existing  SCSTs.  These 
recommendations  addressed  three  aspects  of  the 
training  system:  1)  trainee  information,  2) 
instructor  support,  and  3)  training  system 
management.  The  SMARTTS  characteristics  represent 
the  development  and  application  of  these  earlier 
developed  concepts. 

(91 

Hammed  et  al.'  '  carried  the  Investigation  a 
step  further  by  developing  a  performance 
measurement  approach  for  submarine  tactics 
training.  The  nature  of  the  submarine  tactical 
problem  (i.e.,  decision-nmaking)  has  precluded  the 
development  of  complete  standardized  operating 
procedures,  resulting  in  a  difficult  performance 
measurement  task.  The  developed  technique  is  based 
on  the  use  of  submarine  system  effectiveness  models 
(e.g.,  weapon  systems  effectiveness  models).  The 
model  would  generate  comolex  measures  of  overall 
system  or  subsystem  effectiveness  which  could  be 
related  to  specific  trainee  actions.  Although  the 
operational  forces  were  still  using  the  analog- 
based  fire  control  systems  at  that  time,  tactical 
operations  were  beginning  to  undergo  substantial 
changes,  which  would  be  conducive  to  the  use  of 
this  performance  measurement  approach  (e.g.,  use  cf 
sound  wave  ray  path  analysis  during  approach  and 
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use  of  probability  of  counterdetection).  These 
tactical  changes  coincided  with  the  major  new 
developments  in  sensors  and  fire  control  systems, 
such  as  the  MK  81  weapon  control  console.  These 
factors  have  led  over  the  succeeding  years  to 
acceptance  of  the  training  concepts  and 
capabilities  that  were  to  be  embodied  in  SMARTTS. 

The  Naval  Training  Equipment  Center  took  another 
step  in  the  development  of  submarine  tactics 
training  capabilities  by  evaluating  a  laboratory 
version  of  the  advanced  training  technology 

subsystem^ that  embodied  many  of  the 
recommended  training  assistance  capabilities.  A 
variety  of  Naval  training  and  operations  personnel 
from  Submarine  Forces,  Pacific  (SU8PAC);  Submarine 
Training,  Pacific  (SUBTRAPAC) ;  Submarine  Forces, 
Atlantic  (SUBLANT);  SUBSCOL;  and  other  groups 
reviewed  the  capabilities  of  the  laboratory  version 
and  participated  in  experiments  with  it.  The 
response  was  uniformly  strong  in  approval  of  using 
the  advanced  training  technology  concepts. 

Additional  support  for  the  advanced  training 
technology  concepts  came  from  naval  training 
personnel  at  the  21A40  SCST  site.  The  limited 
training  technology  concepts  incorporated  into  the 
development  of  the  handbook  for  the  21A40  SCST 
resulted  in  requests  by  the  staff  for  the  upgrading 
of  their  trainer  to  include  several  of  these 
capabilities.  The  Naval  Personnel  Research  and 
Development  Center  (NPRDC)  developed  a  limited 
experimental  capability  on  the  21A40  for 
investigating  the  effectiveness  and  acceptance  of 
several  of  the  recommended  training  assistance 
technology  capabilities.  Results  from  the 
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investigation  by  Callan,  kelly,  and  Nicotra'  '  of 
the  operational  training  of  submarine  crews,  showed 
increased  training  effectiveness  as  a  result  of  the 
advanced  training  technology.  Many  of  these 
concepts  have  been  further  incorporated  into  other 
training  programs  and  devices  (e.g..  Sonar 
Operational  Training  and  Assessment  Program  — 
SOTAP);  these  applications  generally  support  the 
effectiveness  of  the  advanced  training  technology 
concepts. 

The  advanced  training  technology  concepts 
forming  the  training  subsystem  that  should  be 
integrated  into  the  training  device/simulator 
design  were  developed  by  a  series  of  NTDC/NTEC 
investigations.  The  systematic  applied  development 
of  the  advanced  training  technology  subsystem, 
however,  remains  to  be  undertaken.  This  is  being 
accomplished  via  the  SMARTTS  program. 

TACTICS  TRAINING  SYSTEM 

The  training  system,  in  any  area  of  application, 
is  composed  of  several  elements.  Various 
specifications  for  these  elements  are  avail¬ 
able^13,  14^.  A  somewhat  simplified  version  of  the 
training  system,  as  shown  in  Figure  1,  consists  of 
four  major  elements.  The  training  objectives, 
which  comprise  the  first  element,  specifically 
define  the  goals  for  which  the  remainder  of  the 
training  system  should  be  designed  in  the  most 
cost-effective  manner.  The  training  objectives  are 
typically  the  result  of  an  analysis  of  tasks, 
skills,  knowledge,  performance  standards,  and 
conditions  under  which  the  preceding  should  be 
attained. 


Figure  1.  Elements  of  the  Training  System 


The  design  characteristics  for  the  remaining 
three  elements  of  the  training  system  should  be 
determined  on  the  basis  of  a  trade-off  analysis 
between  each,  so  as  to  achieve  the  most  cost- 
effective  overall  system  design.  The  training 
program  addresses  the  content  of  the  training 
process  as  well  as  the  methodology  by  which  it  is 
achieved.  The  materials  to  support  the  training 
process  (e.g.,  instructor's  guide,  trainee 
handbook,  and  exercises)  contain  the  training 
program.  The  training  devices  are  tools  to  be  used 
by  the  instructor  as  an  aid  in  accomplishing  the 
training  objectives  during  the  training  program. 
The  training  system  may  have  one  or  more  training 
devices,  including  both  part  task  and  whole  task 
trainers.  On-the-job  training,  the  final  element 
represented  in  Figure  I,  is  an  alternative  to  the 
training  device  in  achieving  certain  of  the 
training  objectives.  It  is  important  that  on-the- 
job  training  be  investigated  as  a  cost/effective 
means  in  trade-off  with  the  training  device. 

Close  Integration  between  the  design 
characteristics  of  the  training  device  and  training 
program  is  of  primary  Importance  to  the  achievement 
of  a  cost-effective  training  system  design.  The 
training  device  should  be  viewed  as  an  aid  to  the 
conduct  of  the  training  program,  in  the  achievement 
of  the  training  objectives.  To  this  end,  the 
characteristics  of  the  training  device  should  be 
designed  so  as  to  aid  the  training  program.  This 
point  is  crucial;  the  training  device  should 
contain  only  those  characteristics  that  most  cost- 
effectively  aid  the  training  process.  The  modern 
training  device,  therefore,  in  the  context  of 
complex  operational  systems  such  as  the  submarine 
combat  system,  should  have  two  major  functions:  1) 
simulation  of  the  operating  environment  and  2) 
training  assistance  (see  Figure  2).  The  vast 
majority  of  simulator-based  training  devices  in  use 
today,  by  the  military  as  well  as  commercial 
industry,  do  not  adequately  address  the  training 
problem.  That  is,  they  typically  provide  adequate 
simulation  but  provide  little  if  any  assistance  to 
the  conduct  of  the  training  process.  The  typical 
complex  simulator/training  device  is  computer  based 
and  has  the  capability  to  generate  a  wide  variety 
of  Information  relevant  to  the  conduct  of  the 
training  process,  as  well  as  provide  much  needed 
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Figure  2.  Major  Subsystems  of  the  Training 
Device  and  Their  Functions 
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assistance  to  the  instructor.  This  aspect  of  the 
training  device  has  been  grossly  overlooked,  al¬ 
though  the  basic  capabilities  may  be  readily  devel¬ 
oped  to  provide  effective  assistance  to  the 
trainees  and  instructor.  SMARTTS  directly  ad¬ 
dresses  the  training  functions  of  the  training 
device;  it  does  not  address  the  simulation  func¬ 
tion.  SMARTTS  keys  on  the  training  system  concept, 
allocating  to  the  instructor  and  other  elements  of 
the  system  those  tasks  for  which  they  are  most 
capable.  For  example,  SMARTTS  relieves  the  in¬ 
structor  from  rudimentary  tasks  such  as  data  re¬ 
cording,  allowing  him  additional  capacity  to  focus 
on  more  important  tasks  of  the  training  process 
(e.g. ,  monitoring  trainee  performance).  Further¬ 
more,  SMARTTS  capabilities  supplement  the  instruc¬ 
tor  by  generating  needed  information  and  providing 
control  assistance  which  are  typically  beyond  the 
capability  of  most  instructors.  Furthermore,  the 
capabilities  are  designed  to  permit  the  evolution 
of  the  training  system  so  as  to  be  responsive  to  the 
constantly  evolving  operational  and  training  needs. 
SMARTTS  is  a  set  of  capabilities  that  represents 
the  training  function  of  the  training  device,  in 
support  of  the  training  process. 

SMARTTS  DESCRIPTION 

SMARTTS  will  be  the  first  major  subsystem  of  a 
simulator-based  training  device  developed  specifi¬ 
cally  to  support  the  training  function,  rather  than 
the  simulation  function  of  the  training  device.  An 
extensive  requirements  analysis,  therefore,  was  ne¬ 
cessary.  This  analysis  drew  heavily  from  previous 
investigations  to  define  the  tasks  of  the  various 
members  of  the  fire  control  and  sonar  parties  on 
the  SSN  and  SSBN.  The  available  information  was 
updated  as  necessary  to  address  the  tasks  to  be 
performed  in  support  of  the  MK  117  and  Nt  113  Mod  9 
fire  control  systems,  and  the  AN/BQQ-5  and  AN/BQQ-2 
sonar  systems,  and  the  utilization  of  recently 
developed  weapon  systems.  The  analysis  also  iden¬ 
tified  skill  and  knowledge  requirements  as  well  as 
performance  criteria,  measures,  and  standards,  and 
Identified  the  comprehensive  set  of  training  objec¬ 
tives  across  submarine  tactics  training.  The  func¬ 
tional  characteristics  for  the  SMARTTS  were  devel¬ 
oped  in  consonance  with  these.  Additionally,  hard¬ 
ware  and  software  design  aspects  of  the  potential 


SMARTTS  were  analyzed  during  this  analysis  so  as  to 
delineate  the  potential  costs  of  SMARTTS.  The  in- 
depth  requirements  analysis  was  necessary  for 
SMARTTS  since  it  represents  the  initial  attempt  to 
comprehensively  define  the  training  function  of  the 
training  device.  This  represents  a  substantial 
change  in  philosophy  regarding  the  design  of  the 
training  device;  that  is,  a  change  from  simulator 
design  to  simulator  plus  training  technology  de¬ 
sign.  It,  furthermore,  represents  the  evolution  of 
the  training  device  from  merely  a  support  aid  into 
a  central  integrating  function  directly  impacting 
the  design  and  conduct  of  many  aspects  of  the 
training  process. 

SMARTTS  provides  the  foundation  upon  which  the 
Integration  of  advanced  training  technology  into 
the  training  device  will  have  been  developed  and 
evaluated.  Results  of  the  test  and  evaluation 
phase  of  SMARTTS,  in  the  operational  environment, 
will  complete  the  initial  development.  It  will 
result  in  a  catalog  of  demonstrated  training  tech¬ 
nology  concepts  that  can  be  integrated  into  other 
training  devices  in  a  wide  range  of  applications. 
Subsequent  applications  of  this  training  technology 
will  require  a  considerably  less  extensive  training 
requirements  analysis  since  the  concepts  are  gener¬ 
ic  in  nature  and  may  be  readily  tailored  to  many 
training  applications.  Rather,  the  development  of 
a  new  training  device  would  require  an  analysis 
only  to  identify  the  training  objectives  and  to 
select  from  those  SMARTTS  characteristics  that  have 
already  been  developed.  The  selected  characteris¬ 
tics  would  have  to  be  tailored  to  the  particular 
training  application;  this,  however,  would  require 
considerably  less  effort. 

SMARTTS  will  provide  features  to  support  three 
major  aspects  of  the  training  process  (see  Figure 
3):  1)  trainee  information,  2)  Instructor  support, 
and  3)  training  system  management.  SMARTTS  capabi¬ 
lities  in  each  of  these  areas  are  summarized  below. 

Trainee  Information 


The  importance  of  feedback  information  to  the 
trainee  regarding  the  outcome  of  his  actions  has 
been  known  for  many  years  as  an  essential  part  of  an 
effective  training  process.  Relatively  little  in- 


FI  gure  3.  Major  Support  Areas  of  SMARTTS 
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formation  Is  currently  provided  to  the  trainees 
prior  to,  during,  or  even  after  the  training  exer¬ 
cise  on  the  SCST.  The  information  that  is  provided 
to  the  trainee  is  generally  a  result  of  the  in¬ 
structor's  observations,  and  is  provided  verbally 
immediately  following  the  exercise.  SMARTTS  will 
provide  capabilities  to  generate,  store,  and  pre¬ 
sent  performance-related  information  to  the 
trainees.  A  wide  variety  of  tactical  and  perfor¬ 
mance-related  parameters  will  be  computed  during 
the  exercise;  these  may  also  be  computed  prior  to 
and  following  each  exercise  so  as  to  discuss  re¬ 
lated  points  and  alternative  ownship  actions.  The 
generated  parameters  will  be  presented  to  the 
trainee  on  appropriately  designed  information  dis¬ 
plays  (e.g.,  graphical,  tabular  summaries). 
SMARTTS  will  also  provide  the  instructor  with  the 
capabilities  to  present  the  information  in  a  vari¬ 
ety  of  formats,  tailored  to  the  specific  training 
problem.  The  information  may  be  presented  to  the 
trainees  on  CRT  monitors  in  the  attack  center 
(e.g.,  during  a  problem  freeze)  and  on  large  screen 
displays  in  an  adjoining  classroom  (i.e.,  prior  to 
and/or  following  the  simulator  exercise).  An  exam¬ 
ple  of  a  graphical  display  presenting  several  tac¬ 
tical  performance  indicators  simultaneously  is  pre¬ 
sented  in  Figure  4.  The  important  aspect  of  these 
characteristics  is  that  SMARTTS  will  provide  the 
capability  to  generate  and  present  the  information. 
The  particular  performance  indicators  used,  display 
formats,  exercises,  etc.,  will  be  tailored  to  the 
particular  training  problem;  these  are  likely  to 
evolve  over  time,  changing  to  meet  the  changing 
training  problem.  SMARTTS  will  provide  meaningful 
visual  feedback  to  the  trainee,  enabling  him  to 
correlate  his  actions  with  relevant  tactical  param¬ 
eters  pertaining  to  his  teammates,  ownship  disposi¬ 
tion,  and  the  target  disposition.  These  character¬ 
istics  represent  the  major  area  of  SMARTTS  concern. 
A  summary  of  the  specific  characteristics  follows: 

•  Preexercise  briefing.  A  variety  of  capabili¬ 
ties  will  be  available  in  a  classroom  context 
to  assist  the  instructor  in  briefing  the 
trainees  prior  to  participating  in  the  simu¬ 
lator-based  exercises.  The  alternative 
tactics  capability  will  enable  fast-time  gen¬ 


Figure  4.  Example  of  Trainee  Single 
Graph  Feedback  Display 


eration  of  problems  in  the  classroom,  includ¬ 
ing  the  generation  and  display  of  performance 
indicators  and  other  tactical  parameters  for 
a  variety  of  ownship  and  target  tactical 
actions.  The  classroom  preexercise  briefing 
sessions  can  be  conducted  while  a  different 
group  is  simultaneously  participating  in 
training  on  the  simulator. 

t  Information  displays.  SMARTTS  will  generate 
a  variety  of  performance  indicators  and  rele¬ 
vant  tactical  parameters  (e.g.,  probability 
of  counterdetection).  The  information  gene¬ 
rated  can  be  presented  to  the  trainees  on 
CRTs  in  the  attack  center  and  on  a  large 
screen  display  in  the  adjoining  classroom. 
These  displays  will  provide  the  means  to 
present  information  for  the  preexercise 
briefing  discussions,  immediate  feedback 
during  the  simulator  exercise,  and  the  post¬ 
exercise  debriefing  discussions.  The  infor¬ 
mation  so  presented  will  enable  the  trainee 
to  learn  and  understand  the  relationships 
between  the  various  tactical  parameters  of 
interest  as  well  as  the  impact  of  his  actions 
and  ownship's  actions  on  the  tactical  prob¬ 
lem.  The  displays  will  be  under  control  of 
the  instructor.  A  trainee  entry  device 
(i.e.,  a  button)  will  be  available  at  each 
operating  station  to  permit  flagging  of  tac¬ 
tical  events  during  the  exercise  for  later 
discussion. 

•  Alternative  tactics.  Alternative  sets  of 
ownship  and  target  actions  can  be  generated 
in  fast-time  during  briefings  to  enable  in¬ 
vestigation  of  the  relationships  between  re¬ 
levant  parameters  (e.g.,  the  impact  of 
earlier  ownship  maneuvers;  the  impact  of  dif¬ 
ferent  ownship  maneuvers  on  target  motion 
analysis  quality).  These  will  assist  train¬ 
ing  by  enabling  direct  comparisons  of  perfor¬ 
mance  indicators  generated  during  the  alter¬ 
native  actions. 

•  Postexercise  debriefing.  The  postexercise 
debriefing  is  currently  used  in  SCST  train¬ 
ing.  The  SMARTTS  information  generation, 
storage,  and  presentation  capabilities  will 
greatly  enhance  this  process.  Postexercise 
debriefing  will  be  possible  in  the  attack 
center  using  the  SMARTTS  CRT  displays. 
Ideally,  however,  the  debriefing  sessions 
will  be  conducted  in  the  classroom  using  the 
large  screen  display.  The  SMARTTS  capabili¬ 
ties  will  enable  the  presentation  of  all 
relevant  information  concerning  the  just- 
completed  exercise  as  well  as  the  generation 
and  investigation  of  alternative  sets  of  tac¬ 
tics  for  the  exercise  situation  and  other 
situations.  The  capability  will  be  present 
for  the  instructor  and  trainees  to  completely 
analyze  the  tactical  problem,  and  focus  on 
aspects  of  particular  interest. 


Instructor  Support 


SMARTTS  will  greatly  augment  the  currently  li¬ 
mited  Instructor  support  facilities  at  each  of  the 
SCSTs.  It  will  provide  the  instructor  with  addi¬ 
tional  capabilities  to  design  and  set  up  exercises, 
to  control  and  monitor  exercises  while  they  are 
being  conducted,  and  to  provide  more  effective 
information  to  the  trainees  prior  to,  during,  and 
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after  each  simulator  exercise.  Instructor  support 
capabilities  are  summarized  below: 

•  Exercise  selection  library.  A  library  of 
highly  structured  prestored  exercises  will  be 
available  for  selection  by  the  instructor. 
Each  exercise  will  be  keyed  to  tactical  and 
training  objectives.  The  exercises  will  be 
highly  structured  allowing  for  necessary  mod¬ 
ification  by  the  instructor  either  prior  to 
or  during  their  conduct  on  the  simulator. 
Furthermore,  the  set  of  available  exercises 
will  be  standardized  across  all  SCST  sites. 

•  Exercise  setup.  An  off-line  capability  will 
be  provided  to  enable  the  instructor  to  de¬ 
velop  and  evaluate  exercises  prior  to  their 
use  in  training.  A  remote  terminal  will  be 
prov'ied  for  this  capability.  The  instructor 
will  be  able  to  develop  the  exercises,  run 
them  in  fast-time,  and  investigate  the  re¬ 
sults.  He  can  designate  appropriate  perfor¬ 
mance  indicators,  training  objectives,  event 
cues,  and  so  on,  which  will  be  prestored  with 
the  exercise.  Furthermore,  this  capability 
will  enable  him  to  compare  different  exer¬ 
cises  and  readily  modify  them  to  achieve 
particular  exercise  design  goals.  Finally, 
the  instructor  will  be  able  to  add  the  new  or 
modified  exercise  to  the  exercise  library  for 
permanent  storage. 

•  Performance  evaluation.  A  set  of  performance 
indicators  will  be  available  under  SMARTTS. 
A  subset  of  these  indicators  will  be  keyed  to 
each  exercise,  achieving  automatic  genera¬ 
tion  and  storage  of  the  performance  data 
during  the  exercise.  The  instructor  may 
develop  and  store  additional  performance  in¬ 
dicators  as  well  as  modify  existing  perfor¬ 
mance  indicators.  Furthermore,  he  can  desig¬ 
nate  additional  performance  indicators  for  a 
particular  exercise  or  remove  existing  per¬ 
formance  indicators  from  an  exercise.  Cues 
will  be  provided,  keyed  to  each  exercise,  to 
alert  the  instructor  to  particular  predeter¬ 
mined  observation  requirements.  The  variety 
of  performance  indicators,  and  other  tactical 
parameters  will  be  available  at  an  instruc¬ 
tor's  remote  console  for  monitoring  during 
the  exercise.  The  information  will  be  pre¬ 
sented  on  a  CRT  display  in  an  appropriate 
format  (i.e.,  graphical,  tabular,  or  some 
combination);  additionally,  selected  param¬ 
eters  will  be  available  for  monitoring  on  an 
instructor's  portable  handheld  remote  con¬ 
sole. 

•  Instructor's  consoles.  A  remote  semiportable 
console  will  be  available  for  monitoring  and 
control  of  the  trainee  exercise.  It  will 
consist  of  a  CRT  display,  general  purpose 
keyboard,  and  function  keyboard.  This  semi- 
portable  device  will  be  easily  moveable  to 
different  locations  in  the  attack  center.  It 
will  enable  the  instructor  to  monitor  and 
control  from  an  attack  center  location  while 
observing  the  trainees.  A  portable  handheld 
display  will  also  he  available  to  enable  the 
Instructor  to  monitor  selected  parameters 
while  moving  about  In  the  attack  center. 
Ideally,  this  portable  display  will  be  wire¬ 
less.  The  Instructor  displays  will  present 
information  related  to  the  performance 


indicators,  current  tactical  parameters, 
geographic  track  history,  ownship  status, 
training  cues,  instructional  alerts,  and 
alarm  conditions. 

•  Training  control.  The  instructor  will  have 
expanded  capabilities  to  control  the  training 
exercise.  These  will  include  a  freeze  capa¬ 
bility,  projection  of  multiple  ownship  and 
target  actions  to  evaluate  their  likely  im¬ 
pact  on  the  exercise,  and  analysis  of  trainee 
progress  to  assist  in  tailoring  exercise  se¬ 
lection  and  configuration. 

•  Instructors'  handbook.  A  comprehensive  in¬ 
structor's  handbook  will  be  developed  to  as¬ 
sist  in  the  development  and  conduct  of  an 
effective  training  process  using  the  SMARTTS 
capabilities.  The  handbook  will  address  the 
particular  capabilities  of  SMARTTS  as  well  as 
effective  training  practices.  An  instruc¬ 
tor's  course  will  be  developed  and  adminis¬ 
tered  to  all  instructors  prior  to  their  use 
of  the  SMARTTS  system. 

•  Automatic  interactive  target.  Currently, 
target  actions  or,  an  SCST  are  either  prede¬ 
termined  and  '-anned,  or  controlled  by  the 
instructor.  Target  control  by  the  instructor 
requires  a  considerable  amount  of  time  and 
often  reflects  the  particular  instructor's 
experiences  rather  than  a  representative  set 
of  target  actions.  The  automatic  interactive 
target  (AIT)  will  provide  for  computer  con¬ 
trol  of  target  actions,  emulating  in  a  proba¬ 
bilistic  manner  likely  actions  of  the  target, 
based  on  tha  best  ave liable  intel licence  in¬ 
formation.  Th?  teroet  i.’i’l,  in  essence, 
react  to  ownship  actions  in  concert  with 
information  provided  via  the  environment,  the 
target  type,  his  mission,  and  the  world  situ¬ 
ation. 

Training  System  Management 

The  preceding  two  areas  of  SMARTTS  primarily 
addressed  conduct  of  the  training  process.  The 
training  system  management  capabilities  of  SMARTTS 
address  the  long-term  structure  and  functioning  of 
the  training  system  and  the  training  process.  It 
provides  for  both  intrasite  and  intersite  manage¬ 
ment  of  the  training  process.  Specific  capabili¬ 
ties  are  as  follows: 

•  Standardized  curriculum  materials.  SMARTTS 
characteristics  and  other  materials  provided 
along  with  SMARTTS  will  be  standardized 
across  all  SCST  sites.  These  will  include 
instructor  and  trainee  handbooks,  the  exer¬ 
cise  library,  the  SMARTTS  system  operator's 
manual,  and  an  integrated  curriculum  across 
sites. 

•  SCST  site  management.  The  tactics  training 
curriculum  will  be  standardized  across  SCST 
sites,  appropriate  to  the  level  of  trainees 
and  other  factors  being  addressed  at  each 
site.  Each  site  will  participate  in  the 
development  of  the  curriculum;  each  site 
will,  furthermore,  have  the  capability  to 
tailor  the  curriculum  to  meet  specific  local 
needs.  Changes  in  tactical  doctrine  would  be 
quickly  implemented  by  a  centralized  activi¬ 
ty,  resulting  in  a  reduced  administrative 
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burden  at  each  site.  Instructor  training 
will  be  standardized  for  SMARTTS  across 
sites.  Finally,  various  training  methods 
will  be  recommended  singly  and  in  combination 
for  use  with  various  exercises;  this  will 
further  standardize  the  training  methodology 
employed  across  sites. 

•  Centralized  tactics  training  system  manage¬ 
ment.  A  training  management  structure  has 
been  identified  to  configure  and  control  the 
training  process  across  SCST  sites.  This 
structure  would  come  under  the  force  com¬ 
mander's  direction  and  would  Drovide  more 
unified  and  controlled  curriculum,  proce¬ 
dures,  and  performance  criteria.  This  struc¬ 
ture  would  act  as  the  coordinating  body 
across  all  SCST  sites.  Furthermore,  the 
effectiveness  of  training  at  each  site,  the 
effectiveness  of  the  various  training  methods 
being  employed,  the  exercises  being  used, 
etc.,  would  be  monitored  by  this  central 
activity,  providing  for  a  long-term  analysis 
of  the  system's  training  effectiveness  and 
the  training  needs. 

SMARTTS  DEVELOPMENT 

The  above  discussion  addresses  the  three  major 
areas  of  SMARTTS,  and  summarizes  characteristics  of 
each  area.  These  characteristics,  which  are  expli¬ 
citly  identified  in  the  SMARTTS  Type  "A"  Specifica¬ 
tion,  pertain  to  the  eventual  production 

SMARTTS.  A  preprototype  version  of  SMARTTS  is 
currently  being  developed  and  installed  on  the 
21A41  SCST  at  Norfolk,  Virginia.  This  preprototype 
will  incorporate  the  essential  features  of  SMARTTS, 
although  not  all  of  the  specific  characteristics. 
It  should  be  noted  that  the  essence  of  SMARTTS  is 
the  general  computer-based  capabilities  denoted  by 
the  three  major  areas;  the  specific  characteristics 
under  each  of  these  areas  are  likely  to  evolve  as 
SMARTTS  is  used,  with  changing  operational  probl¬ 
ems,  changing  training  needs,  and  developing  train¬ 
ing  technology.  The  SMARTTS  capabilities  will 
enable  this  evolution  of  characteristics  to  occur. 

The  preprototype  installation  on  the  21A41  SCST 
will  enable  the  evaluation  of  SMARTTS  in  the  opera¬ 
tional  training  setting.  Information  is  presently 
being  collected  at  the  21A41  SCST  and  other  SCST 
sites  regarding  the  instructor  functions,  training 
methodologies  employed,  and  tactical  team  perfor¬ 
mance  prior  to  and  following  training  sessions. 
These  data  collected  from  traditionally-trained 
groups  will  comprise  the  control  group  data  base 
for  comparison  with  the  SMARTTS-trained  groups 
after  the  preprototype  installation  becomes  opera¬ 
tional.  The  collection  of  SMARTTS  training  data, 
as  well  as  the  subsequent  evaluation  of  the  train¬ 
ing  effectiveness  of  SMARTTS,  will  be  conducted  by 
an  independent  group.  The  purpose  of  this  investi¬ 
gation  will  be  two-fold:  1)  to  evaluate  the  effec¬ 
tiveness  of  groups  trained  using  the  SMARTTS  tech¬ 
nology  with  traditionally  trained  groups,  and  2)  to 
evaluate  and  improve  the  training  effectiveness  of 
the  various  characteristics  of  SMARTTS.  It  is 
expected  that  although  SMARTTS  will  prove  to  be  a 
cost-effective  adjunct  to  the  simulator/training 
device,  its  relatively  large  number  of  characteris¬ 
tics  will  have  differing  degrees  of  training  effec¬ 
tiveness.  Furthermore,  it  is  expected  that  these 
characteristics  will  be  improved  over  time  and  will 
be  supplemented  by  additional  SMARTTS  characteris¬ 


tics  as  appropriate  to  enhance  the  training  pro¬ 
cess.  It  is  the  intent,  therefore,  that  the 
SMARTTS  evaluation  also  generate  information  rele¬ 
vant  to  the  improvement/evolution  of  the  various 
SMARTTS  characteristics. 

SMARTTS  is  a  training  subsystem  specifically 
designed  to  meet  requirements  for  the  tactics 
training  of  submarine  officers.  The  training  tech¬ 
nology  concepts  identified  and  developed,  however, 
embody  considerable  generic  aspects.  Hence,  they 
are  likely  to  be  directly  applicable  to  most  simu¬ 
lator-based  training  systems.  It  is  likely  that 
SMARTTS-type  capabilities  should  be  resident  on 
many  major  computer  controlled  training  devices. 
Specific  characteristics,  such  as  particular  per¬ 
formance  indicators  and  display  formats,  neces¬ 
sarily  have  to  be  tailored  to  each  particular 
training  application.  Nevertheless,  the  generic 
capabilities  represented  by  SMARTTS  are  potentially 
applicable  to  substantially  impact  the  effective¬ 
ness  in  a  wide  range  o.f  training  areas.  For 
example,  the  SMARTTS  technology  has  been  applied 
and  investigated  for  application  in  commercial  deck 
officer  training  on  ship  bridge/shiphandling  simu¬ 
lators;  preliminary  results  indicate  that  this  type 
of  technology  would  substantially  impact  the  effec¬ 
tiveness  of  those  training  devices.  As  an  order  of 
magnitude  estimate,  the  SMARTTS  capabilities,  if 
developed  and  designed  into  the  training  device 
when  initially  procured,  would  cost  about  20  per¬ 
cent  of  the  device  cost.  Since  the  characteristics 
are  predominantly  implemented  in  software,  rela¬ 
tively  minor  additional  cost  increments  would  be 
required  for  additional  production  units. 

The  training  device,  albeit  a  simulator,  has 
often  been  developed  independently  of  the  other 
elements  in  the  training  system.  The  potentially 
extensive  training  assistance  capabilities  of  the 
training  device/simulator  when  configured  to  pro¬ 
vide  the  SMARTTS-type  training  technology,  necessi¬ 
tate  close  integration  with  the  other  elements  of 
the  training  system  so  as  to  achieve  the  most 
effective  design.  Whereas  the  training  systems 
design  approach  (e.g..  Instructional  Systems  Devel¬ 
opment  approach  has  often  been  ignored  in  the  past, 
the  potential  training  effectiveness  gains  to  be 
achieved  by  incorporating  the  SMARTTS-type  techno¬ 
logy  make  the  systems  approach  mandatory.  The 
training  device  should  not  be  independently  devel¬ 
oped;  rather,  it  must  be  developed  and  closely 
integrated  with  the  training  program  to  meet  the 
specific  training  objectives. 
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ABSTRACT 


For  certain  weapons  systems,  an  all-software  implementation  of  the  operator  proficiency 
trainer  is  possible.  The  prospective  benefit  is  reduction  of  unit  production  costs.  Suitable 
systems  for  this  approach  are  generally  identifiable  by  their  use  of  computer  display  and  con¬ 
trol  consoles  as  operator  stations.  The  system' s  basic  equipment  configuration  may  appear 
to  impose  limitations  for  training  usage,  such  as  lack  of  an  instructor  station  or  a  computer 
memory  with  insufficient  capacity  for  storing  the  data  of  a  realistic  simulation  scenario,  but 
in  a  cost-effectiveness  appraisal  of  the  all-software  approach,  the  possibilities  for  software 
solutions  to  these  hardware  limitations  should  be  considered.  This  paper  reviews  these  con¬ 
siderations  and  the  design  decisions  for  a  specific  case,  the  Troop  Proficiency  Trainer  of  the 
U.  S.  Army's  PATRIOT  Air  Defense  System.  In  this  trainer,  simulation  driver  software  is 
combined  with  tactical  computer  programs  to  produce  highly  realistic  training  exercises,  and 
considerable  flexibility  is  achieved  for  scenario  selection  and  other  aspects  of  exercise  zontrol. 


WEAPONS  SYSTEM  OPERATION 

PATRIOT  is  a  guided  missile  system  designed 
to  counter  the  threat  to  field  army  air  defense 
through  the  1980' s  and  1990' s.  The  threat  is 
characterized  by  defense  suppression  tactics  of 
maneuver,  electronic  countermeasures  and  numer¬ 
ical  saturation.  One  of  the  key  features  in  the 
PATRIOT  response  to  this  threat  is  automated 
operation  with  provisions  for  human  override.  O) 

The  Engagement  Control  Station  (ECS)  is  the 
center  of  operational  control  in  the  PATRIOT  Fire 
Unit  (Figure  1)  and  is  the  only  manned  station  in 
the  Fire  Unit  during  battle.  It  houses  operators, 
a  weapon  control  computer,  man-machine  inter¬ 
face  and  various  data  and  communications  termi¬ 
nals.  Operational  control  at  the  ECS  involves 
radar  surveillance,  target  evaluation,  launch 
decision,  weapon  assignment,  missile  guidance, 
Fire  Unit  status  monitoring  and  communications. 

Another  unit  which  is  manned  during  battle  is 
the  PATRIOT  Command  and  Coordination  Set 
(CCS).  The  CCS  (Figure  2)  is  the  operational  con¬ 
trol  center  for  coordinating  PATRIOT  firepower 
with  other  users  of  the  airspace.  It  interfaces 
with  U.  S.  Army  air  defense  group  facilities, 
neighboring  CCSs  and  with  several  ECSs.  It 
houses  operators,  a  weapon  control  computer, 
man/machine  interface,  and  various  data  and 
communications  terminals.  The  primary  physical 
difference  between  the  CCS  and  the  ECS  is  the 
absence  of  radar  interface  and  launcher  communi¬ 
cations  equipment  in  the  CCS. 

In  both  the  ECS  and  CCS,  human  override  of 
automated  operations  is  facilitated  by  a  man- 
machine  interface  consisting  of  displays  and  con¬ 
trols  hardware  (Figure  3)  and  associated  soft¬ 
ware.  The  displays  and  controls  design  includes 
two  operator  consoles:  console  mode  selections 
enable  the  division  of  operator  tasks.  System 
status  information  is  displayed  in  tabular  format 
and  on  indicator  panels.  Cathode-ray  tubes  dis¬ 
play  the  air  defense  situation,  related  tabular 
data,  and  alert  messages.  The  operators  moni¬ 
tor  system  operation  via  these  displays  and  enter 


override  commands  and  data  via  console  switches 
and  keys.  Clearly,  these  operators  require  con¬ 
tinuing  practice  in  realistic  training  exercises  in 
order  to  maintain  and  enhance  their  proficiency 
and  readiness. 

EMBEDDED  TRAINER  REQUIREMENTS 

Refresh  training  is  a  requirement  for 
PATRIOT  system  operators.  It  prescribes  a 
capability  to  simulate  enemy  air  attacks  on 
assets  defended  by  the  system,  with  modeling  of 
the  effects  of  maneuver,  ECM,  weather  and  other 
environmental  influences,  combined  with  a  means 
for  evaluating  operator  responses  to  such  attacks. 
It  further  states  that  each  prime  item  of  equip¬ 
ment  (ECS  and  CCS)  shall  incorporate  this  train¬ 
ing  capability  and  be  self  sufficient  for  maintain¬ 
ing  operator  proficiency.  The  training  require¬ 
ment  also  implies  an  ability  to  train  without 
emitting  radiations,  either  radar  or  radio.  The 
major  safety  requirement  is  to  prevent  inadvert¬ 
ent  missile  launch. 

ALL-SOFTWARE  DECISION 

In  design  studies  performed  prior  to  imple¬ 
mentation,  several  factors  influenced  the  hard¬ 
ware/software  allocation  of  trainer  functions. 

One  constraint  was  available  space;  there  was 
virtually  no  place  left  in  the  ECS  and  CCS  shelter 
for  installing  additional  equipment.  However, 
considerable  hardware  capability  was  already 
present  in  existing  displays  and  controls  (Figure 

3)  and  the  tactical  computer  configuration  (Figure 

4) .  The  alternative  of  using  the  tactical  configu¬ 
ration  of  computer  and  displays  appeared  highly 
consistent  with  the  need  for  realism  in  training 
exercises.  The  fact  that  computer-synthesized 
data  is  displayed  (target  symbols  vice  radar 
return  analog  representations,  etc.  )  meant  that 
display  images  of  realistic  quality  could  be  gener¬ 
ated  by  the  computer,  without  the  use  of  stimula¬ 
tion  equipment.  With  two  operator  consoles, 
instructor  participation  in  training  exercises  was 
possible.  With  magnetic  tape  for  off-line  storage 
of  programs  and  scenarios,  complete  isolation  of 
the  trainer  from  tactical  software  was  possible. 
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launching  stations 


Figure  1  -  PATRIOT  Fire  Unit 
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Figure  Z  -  Cutaway  View,  Command  and  Coordination  Set 


Figure  3  -  Forward  Find  of  FCS  Shelter  Interior 
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Figure  4  -  Weapon  Control  Computer 
and  Peripherals 

In  what  is  believed  to  be  the  first  such  instance 
among  army  air  defense  systems,  the  decision 
was  made  to  implement  an  embedded  refresh 
trainer  capability  entirely  in  software  for  existing 
tactical  hardware.  The  result  is  the  PATRIOT 
Troop  Proficiency  Trainer  (TPT). 

PATRIOT  TPT  DESIGN 

In  overview,  the  PATRIOT  Troop  Proficiency 
Trainer  involves  both  off-line  and  on-line  data 
processing  (Figure  5).  Data  preparation,  the 
function  of  TFT  off-line  data  processing,  is  per¬ 
formed  with  the  aid  of  three  software  subsystems 
including  the  TPT  Preprocessor.  On-line  trainer 
utilization,  employing  the  existing  tactical  hard¬ 
ware  of  an  ECS  or  CCS,  requires  one  of  two  soft¬ 
ware  subsystems,  the  ECS  or  CCS  TPT. 

Preprocessor 

The  TPT  Preprocessor  makes  real-time 
simulation  exercises  possible  in  the  ECS  TPT 
and  CCS  TPT  by  reducing  the  on-line  processing 
load  and  the  on-line  computer  memory  require¬ 
ments.  The  Preprocessor  reads  input  script 
statements  which  control  data  base  maintenance, 
scenario  generation,  and  output  functions.  The 
data  bases  created  and  maintained  by  the  software 
contain  flight  profiles  (a  maximum  of  50  per  pro¬ 
file  data  base);  performance  characteristics  by 
aircraft  or  missile  type  (a  maximum  of  64  types 
per  characteristics  data  base);  locations  of  ECS 
and  CCS  sites,  with  Fire  Unit  radar  oreintations; 
geographic  coordinates  of  all  locations  of  interest 
(a  maximum  of  400  locations  per  location  identi¬ 
fier  data  base);  and  a  model  of  local  terrain  ele¬ 
vations,  (with  a  maximum  1980  points).  These 
data  base  capacities  are  parameter  values  based 
on  host  computer  (1108)  memory  limitations. 
These  data  bases  are  referenced  during  scenario 
generation.  As  many  as  180  data  bases  (any  mix) 
may  be  stored  on  one  tape. 


Other  scripted  events  include  communications 
messages;  jamming  events;  track  breaks  due  to 
environmental  effects;  and  damage  to  defended 
assets.  A  flight  path  is  forced  into  terrain  follow¬ 
ing  at  a  specified  height  above  ground  level  if  it  is 
scripted  to  fly  below  that  height  (take-offs,  land¬ 
ings  and  impacts  excepted).  The  terrain  model  is 
also  used  by  the  Preprocessor  for  determining 
occultation  or  masking  of  flights  in  ECS  TPT 
scenarios.  Geometrical  processing  of  a  scenario 
eliminates  data  beyond  the  sector  bounds  of  the 
specific  ECS  or  CCS  site  and  converts  coordinates 
to  that  sector' s  coordinate  system.  Scenario  pre¬ 
processing  also  includes  position/velocity  calcula¬ 
tion  along  flight  paths  at  fixed -interval  time  steps, 
and  the  time  ordering  of  all  scenario  data.  The 
output  function  of  the  TPT  Preprocessor  produces 
error  listings,  data  base  listings,  data  base  tapes, 
scenario  listings  and  scenario  tapes.  The  Pre¬ 
processor  runs  on  a  UNIVAC  1)08  computer  sys¬ 
tem,  a  choice  based  on  the  availability  of  an  exist¬ 
ing  Preprocessor  that  was  modified  for  TPT. 

Other  Off-Line  Functions 

Scenarios  are  transferred  from  nontactical 
tape  to  tactical  peripheral  tape  by  a  PATRIOT 
utility  routine  that  runs  on  a  special  system. 

Data  base  tapes  for  TPT  exercises  may  be 
prepared  in  the  field  and,  accordingly,  this  process 
may  be  considered  part  of  the  training  exercise 
even  though  tactical  software,  rather  than  TPT 
software,  is  involved.  The  data  base  produced, 
including  any  errors,  is  input  to  the  TPT  exercise 
and  subsequently  recorded  in  the  evaluation  data 
output  of  the  ECS  or  CCS  TPT. 

On-Line  Software 

The  software  of  the  ECS  TPT  or  CCS  TPT 
operates  in  three  phases:  Initialization,  Operator 
Exercise,  and  Evaluation  (Figure  6). 

Initialization  Phase 


During  initialization,  TPT  software  is  loaded 
into  the  WCC  from  tape,  then  a  data-entry  type  of 
tabular  display  is  presented  at  the  operator  con¬ 
soles.  Guided  by  its  format  the  Training  Officer 
keys  in  exercise  identifying  data  (operator  crew, 
site  ID  and  date:  scenario  identification  is  auto¬ 
matically  recorded)  followed  by  exercise  control 
parameter  values.  These  specify  the  subset  of 
assets  to  be  defended;  kill  efficiency  (a  factor  con¬ 
tributing  to  probability  of  kill  in  the  kill  determina¬ 
tion  algorithm);  scenario  duration  (full,  or  an  early 
stop  time);  the  subset  of  scenario  flight  paths 
(range  of  numbers  and  odd/even  selection);  and 
ECM  permit/suppress.  All  of  these  parameters 
affect  the  difficulty  of  the  training  exercise  and 
they  facilitate  exercise  problem  variations  with  a 
single  scenario.  The  Training  Officer  next  sets 
the  initial  simulated  missile  inventory,  which  will 
be  decremented  during  the  exercise.  Map  displays 
from  the  data  base  are  available  to  him  at  this 
point  in  the  sequence,  and  panel  indicators  display 
status  data,  as  aids  in  briefing  the  operator  crew 
for  the  training  exercise. 


Scenarios  produced  by  the  Preprocessor  con-  Operator  Exercise  Phase 
sist  mainly  of  flight  paths.  These  are  scripted  by 

a  choice  of  several  methods,  including  point  to  Operator  exercise  software  is  driven  by  a 

point,  speed/heading,  profile,  and  related  flights.  simulation  control  function  which  processes  data 
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Figure  6  -  ECS  and  CCS  TPT  Processing  Phases 


from  the  scenario  tape  in  time  order.  Some  of 
the  tactical  software  program  units,  particularly 
those  which  service  the  displays  and  controls, 
are  incorporated  in  TPT  for  the  assurance  of 
realistic  system  response.  The  remaining  tacti¬ 
cal  software  functions  are  modified  by  the  TPT 
software,  retaining  essential  aspects  and  support¬ 
ing  the  simulation.  This  use  of  simplified  models 
of  selected  tactical  software  functions  bypasses 
actual  radiation;  controls  input  variables;  and  has 
the  desirable  side-effect  of  making  WCC  memory 
space  available  for  additional  TPT  software  func¬ 
tions.  Reactivity  of  the  simulation  is  limited  to 
kill  effects;  if  the  software  assesses  a  kill  then 
the  flight  formation  size  is  decremented  by  one, 
and  subsequently  all  related  displays,  communica¬ 
tions  messages,  decoy  launches,  air-to-surface 
missile  launches,  and  asset  damage  events  are 
inhibited  accordingly.  In  the  event  that  the  last 
missile  is  expended  and  it  completes  its  flight, 
or  upon  100  percent  damage  to  the  defending  CCS 
or  ECS,  the  operator  exercise  terminates  pre¬ 
maturely  and  the  remainder  of  the  scenario 
advances  at  an  accelerated  rate  with  only  the 
damage  events  being  processed.  Otherwise  the 
exercise  terminates  at  its  scheduled  completion. 
During  the  exercise  the  software  saves  all  oper¬ 
ator  inputs,  including  time  of  occurrence,  switch 
or  key  action,  and  the  operator  console  identifier. 
These  are  used  during  the  evaluation  phase.  An 
exercise  may  be  conducted  with  no  operator 
inputs,  resulting  in  a  fully -automatic  benchmark 
run  which  provides  the  Training  Officer  with  one 
standard  against  which  to  evaluate  operator 
performance. 

Evaluation  Phase 

Immediately  following  the  exercise  (i.  e. ,  at 
early  stop  time  or  upon  scenario  completion, 
whichever  was  selected  during  the  initialization 
phase)  a  final  score  is  computed  and  displayed. 
Ranging  in  value  from  0  to  100,  this  score 
measures  the  survival  of  defended  assets.  Its 
formula  is 

N  N 

FINAL  SCORE  =  <£  (l00-Da)(W  ))/Z  Wa, 

a=]  a= 1 

where 

a  is  one  of  the  N  assets  selected  during 

initialization, 


D  is  limited  to  a  maximum  value  of  100 
and  is  the  cumulative  percentage  of  damage 
assessed  against  an  asset  during  the  oper¬ 
ator  exercise  phase,  and 

W,  a  weighting  factor,  is  the  asset' s  value 
which  is  obtained  from  the  tactical  soft¬ 
ware  data  base. 

For  the  final  score  to  be  less  than  100,  there  must 
exist  scripted  damage  events  in  the  scenario,  and 
the  damage -perpetrating  flights  must  successfully 
pass  several  filters,  including  the  geographic 
sector  selection  of  flight  paths  performed  by  the 
preprocessor;  exercise  duration  and  simulation 
reference  number  (SRN)  subset  selection  per¬ 
formed  during  initialization;  and  kill  determina¬ 
tions  resulting  from  engagements  during  the  exer¬ 
cise.  Scripted  for  each  damage  event  are  time  of 
occurrence,  SRN  of  the  perpetrating  flight,  identi¬ 
fication  of  asset  to  be  damaged,  and  a  damage  per¬ 
centage  increment.  If  the  flight  remains  in  the 
exercise  at  the  time  of  occurrence,  then  the  incre¬ 
ment  is  multiplied  by  the  number  of  vehicles 
remaining  in  the  flight,  and  the  product  is  added 
to  the  cumulative  percentage  of  damage  (D)  for  the 
asset  identified.  By  killing  flights  that  threaten 
defended  assets  before  damage  occurs,  the  oper¬ 
ator  team  improves  its  final  score. 

Playback  of  the  entire  exercise  is  one  selec¬ 
tion  available  during  the  evaluation  phase.  The 
software  accomplishes  playback  by  resetting  the 
random  number  seed  for  kill  determination;  play¬ 
ing  the  scenario  tape  over  again;  injecting  saved 
operator  actions  at  the  times  indicated;  and  ignor¬ 
ing  real  operator  inputs. 

Other  selections  that  may  be  made  at  the  oper¬ 
ator  consoles  during  the  evaluating  phase  are  data 
recording  on  tape  (including  identifying  data;  exer¬ 
cise  control  parameters;  data  base;  and  the  record 
of  operator  actions),  and  printouts  of  three  differ¬ 
ent  lengths,  containing  some  or  all  of  the  data  that 
is  recorded  on  tape. 

These  evaluation  aids,  the  final  score,  the 
printout  of  data,  and  the  playback  capability,  are 
intended  for  the  use  of  a  Training  Officer  in  evalu¬ 
ating  performance  relative  to  factors  such  as 
friendly  aircraft  protection  and  number  of  missiles 
expended,  as  well  as  the  asset  survival  factor. 
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IMPLEMENTATION  EXPERIENCE  DATA 


The  TPT  development  phase,  including  soft¬ 
ware  requirements  definition  through  integration 
and  operational  testing,  was  completed  in  January, 
1980.  Production  of  additional  TPT  units  involves 
only  the  copying  of  software  and  documents. 

TPT  and  PATRIOT  tactical  software  were 
developed  simultaneously,  with  selected  tactical 
program  units  transferred  and  integrated  into  TPT 
(Figure  7).  Simultaneous  development  and  the 
integration  of  untested  tactical  program  units  com¬ 
plicated  TPT  debugging.  However,  the  availability 
of  TPT  provided  an  unexpected  benefit  when  it  was 
discovered  that  TPT  was  an  excellent  simulation 
test  bed  for  checking  out  and  debugging  tactical 
programs  for  target  evaluation,  launch  decision 
and  weapon  assignment.  The  use  of  selected  tacti¬ 
cal  program  units  in  TPT  reduced  development 
cost,  enhanced  the  realism  of  system  response, 
and  is  expected  to  simplify  the  updating  of  TPT 
when  the  associated  tactical  functions  change. 

The  combined  size  of  the  TPT  software, 
including  Preprocessor,  ECS  and  CCS  TPT,  but 
excluding  integrated  tactical  program  units,  is 
approximately  32  thousand  lines  of  source  code. 

DISCUSSION 

The  PATRIOT  Troop  Proficiency  Trainer  may 
be  characterized  by  several  previously  mentioned 
attributes,  including: 

•  Software  implementation 

•  Use  of  existing  equipment 

•  Absence  of  remote  instructor  console 

•  Negligible  unit  production  costs 

•  Real-time  simulation 

•  Large-scale,  realistic  simulation 

•  Limited  reactivity  (kill  effects  only) 

•  Ease  of  updating 

Performance-related  design  decisions  were 
concerned  mainly  with  the  unloading  of  on-line 
scenario  processing  functions  to  off-line  pre¬ 
processing  in  order  to  assure  a  real-time  simu¬ 
lation  update  rate.  The  results  are  a  scenario 


capacity  of  approximately  2000  target  minutes 
(average  track  load  times  scenario  duration),  real¬ 
istic  target  maneuvers  and  environmental  effects, 
capacities  for  number  of  multiple  simultaneous 
engagements  and  number  of  targets  simultaneously 
tracked  equalling  those  of  the  PATRIOT  tactical 
system,  all  at  a  real-time  simulation  rate.  The 
previously  described  limitation  of  simulation 
reactivity  to  the  area  of  kill  effects  does  not  repre¬ 
sent  a  permanent  constraint;  additional  reactivity 
is  possible  through  software  modification. 

TPT  training  exercises  are  repeatable.  Some 
variations  are  possible  through  exercise  parameter 
selections  during  initialization,  but  control  on  this 
source  of  variation  is  established  by  recording 
parameter  values  in  the  TPT  evaluation  outputs. 
Control  of  variables  is  further  extended  by  use  of 
a  prescripted  scenario  as  the  main  source  of  data 
for  the  training  exercise;  limitation  of  on-line 
reactivity;  and  limitation  of  random-number  usage 
to  the  kill  determination  algorithm.  As  a  result, 
TPT  training  exercise  results  may  be  used  in  com¬ 
parative  evaluations  (Skill  Qualification  Tests  for 
example)  and  as  the  basis  for  statistical  analysis. 

A  most  significant  advantage  of  software  for 
trainer  implementation  is  ease  of  accommodating 
training  objective  changes  or  other  changes  of  a 
basic  nature.  For  example,  the  Army’ s  PATRIOT 
Project  Manager,  Maj.  Gen.  Oliver  D.  Street  III 
recently  wrote  that  new  technology  and  system 
capabilities  call  for  a  review  of  current  air -defense 
doctrine,  which  is  damage -limiting  and  tends  to 
concentrate  on  protecting  critical  assets.  2  This 
doctrine  is  reflected  in  TPT' s  final  score,  which 
rewards  PATRIOT  operators  for  minimizing 
damage  to  their  defended  assets.  If  air  defense 
objectives  were  to  change  to  area  defense  and 
zone -penetrating  raid  attrition,  only  four  TPT 
software  changes  would  be  required:  selection  of 
a  zone  rather  than  assets  during  initialization, 
replacement  of  damage  counting  with  penetration 
counting,  change  of  the  final  score  to  a  measure¬ 
ment  of  raid  attrition,  and  replacement  of  TPT1  s 
integrated  tactical  program  units  that  perform 
target  evaluation  with  updated  versions.  This 
change  in  TPT' s  training  objective  would  be 
accommodated  simply  by  changing  the  software  tap 
tape  that  loads  the  trainer  into  the  tactical 
equipment. 


TACTICAL  SOFTWARE 


•  Real  Time  Executive 

•  Displays  and  Controls 

Information  Processing 


TPT  SOFTWARE 

•  Initialization 

•  Exercise  Phase  Control 

•  Real  Time  Executive 

•  Displays  and  Controls 

Information  Processing 

•  Target  Evaluation,  Launch  Decision 

and  Weapon  Assignment  Logic 


Figure  7  -  Software  Functions  Correspondence,  ECS  Case 
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ADVANCED  FIGHTER  AVIONICS  SIMULATION  DESIGN:  THE  SIMULATE/STIMULATE  QUESTION 
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ABSTRACT 

In  the  real-time  simulation  of  advanced  fighter  avionics  systems,  a  critical 
design  decision  Is  the  choice  of  stimulating  the  actual  aircraft  hardware  sub¬ 
systems  or  simulating  the  subsystem  function  via  general  purpose  computer.  This 
case  study  of  a  design  decision  addresses  the  effects  on  many  aspects  of  the  air¬ 
craft  training  or  research  device;  training  features/research  capability,  reli¬ 
ability  and  maintainability  as  well  as  major  Impact  on  the  development/del 1 very 
schedule. 


INTRODUCTION 

The  Fire  Control  Computer  (FCC)  of  the  F-16 
avionics  suite  Is  the  primary  processor  and 
system  data  bus  (multiplexed  serial  data)  con¬ 
troller.  The  successful  and  timely  Integration 
of  an  F-16  avionics  simulation  Is  therefore 
keyed  to  this  critical  on-board  computation/ 
communication  system. 

Due  to  the  high  technical  risk  of  the  complex 
avionics  simulation  and  schedule  restraints, 
three  major  design  approaches  were  pursued.  The 
first  was  the  utilization  of  the  actual  on-board 
Fire  Control  Computer  system  and  the  stimulation 
of  this  device  by  supplying  all  external  Inputs 
normally  received  In  the  aircraft.  Another 
design  pursued  was  the  translation  of  the  soft¬ 
ware  program  called  the  Operational  Flight 
Program  normally  run  In  the  Fire  Control 
Computer  from  JOVIAL  Into  FORTRAN,  which  could 
then  be  Integrated  with  the  flight  simulation  In 
the  simulator  general  purpose  computer.  The 
third  design  approach  which  was  followed  to 
successful  completion  was  to  simulate  the  Fire 
Control  Computer's  functions  based  on  the  opera¬ 
tional  specifications  and  derived  primarily  from 
the  equation  derivation  level  of  aircraft  system 
documentation.  This  design  selection  represents 
a  major  departure  from  present  Industry  practice. 
The  reduced  cost  and  Improved  training/research 
capability  as  well  as  significantly  reduced 
Implementation  schedule  Indicate  that  the  auto¬ 
matic  acceptance  of  aircraft  hardware  engineers 
and  manufacturers  suggested  techniques  may  be 


adversely  affecting  key  high-technology  design 
decisions  In  training/research  simulation 
devices. 

BACKGROUND 

F-16  Simulator  Program 

The  Advanced  Simulator  for  Pilot  Training 
(ASPT)  F-16  simulation  program  was  designed 
after  the  successful  A-10  "phased"  conversion  of 
an  existing  T-37  cockpit.  The  F-16  phase  I 
effort  was  comprised  of  minimum  objectives 
directly  comparable  to  those  of  the  A-10  program 
(1),  with  a  prioritized  list  of  Phase  I  goals 
which  markedly  extended  the  simulator's 
capability. 

The  F-16  Avionics  System  was  a  critical  high- 
risk  component  of  the  program.  The  goals  for 
this  system  simulation  consisted  of  a  prioritized 
list  (generated  by  Tactical  Air  Command)  of 
training  capabilities.  This  list  consisted  of 
specific  navigation  and  air-to-ground  (A/G) 
delivery  modes  via  the  Heads  Up  Ofsptay  (HUO)  as 
well  as  physical  control  panel  functions  such  as 
the  Stores  Control  Panel  and  Fire  Control  Naviga¬ 
tion  Panel.  Priorities  were  based  on  projected 
training  schedule  and  needs. 

Research  requirements  Included  all  of  ASPT's 
conventional  advanced  training  features.  These 
include  “Freeze",  "Set",  Reset",  weapons  release 
condition  Capture  and  Recall,  "Record",  "Play¬ 
back",  and  specially  configured  tactical  condi¬ 
tions.  Special  features  for  avionics  system 
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training  were  anticipated  such  as  on  "Freeze" 
target  Identification  and  target  tracking  Infor¬ 
mation  based  on  Heads  Up  Display  (HUD)  symbology. 
Other  special  control  features  Mere  also  antici¬ 
pated,  though  not  defined. 

F-16  Avionics  System 

Although  the  primary  design  decision  to  be 
discussed  Is  the  Fire  Control  Computer  (FCC) 
simulate  versus  stimulate  question,  some  back¬ 
ground  of  the  aircraft  avionics  system  Is  helpful 
for  a  basic  understanding  of  the  problem. 

The  F-16  avionics  system  consists  of  nine 
primary  subsystems  Mith  a  common  high  speed 
multiplexed  serial  data  bus  for  digital  conmunl- 
catlon.  The  Fire  Control  Computer  (FCC)  serves 
as  the  controller  for  all  digital  communications 
betMeen  subsystems  and  is  also  the  primary  con¬ 
troller  and  processor  •  for  most  of  the  advanced 
avionics  features  available  on  the  F-16.  All 
air-to-ground  and  air-to-air  ordlnace  delivery 
modes,  except  manual  bombing  and  snapshoot 
gunnery  mode,  are  implemented  in  the  FCC.  Heads 
Up  Display  (HUD)  navigation  Is  also  supported  by 
the  FCC. 

The  Fire  Control  Computer  (FCC)  Is  a  moder¬ 
ately  high  speed  16-bit  digital  mini-computer 
built  by  Delco.  The  Delco  "Magic  362-F"  (FCC) 
has  32K  bytes  of  random  access  core  memory  and 
has  floating  point  arithmetic  Instructions.  The 
FCC  Interfaces  Mith  the  rest  of  the  avionics 
system  primarily  through  Its  Ml  1 -Std-1553  serial 
data  interface,  although  It  also  has  analog  and 
discrete  channels. 

AVIONICS  SYSTEM  DESIGN  ALTERNATIVES 
Stimulate  the  Aircraft  HardMare 

This  approach  strikes  a  warm  spot  In  the 
hearts  of  pilots  and  aircraft  hardware  manu¬ 
facturers  alike.  The  key  potential  advantage 
with  stimulating  the  F-16  FCC  Is  that  the  on¬ 
board  operational  flight  program  (OFF)  might  be 
used  without  modification.  This  should  result 
In  excellent  simulation  fidelity  as  well  as  the 
capability  to  easily  update  to  new  software 
changes  In  the  aircraft  (block  updates).  This 
approach  would  also  appear  to  minimize  simulator 
software  and  hardware  since  the  aircraft  FCC  will 
control  the  avionics  system.  These  potential 
advantages  had  a  number  of  Implementation  short¬ 
falls. 

Stimulation  Implementation 

The  conventional  "stimulate"  approach  was 
considered  the  lowest  risk  approach  and  was 
therefore  a  primary  early  effort.  The  hardware 


configuration  to  support  this  approach  had 
already  been  defined  by  the  Air  Force  Avionics 
Lab  (AFAL)  at  Wright  Patterson  AFB  as  well  as 
the  In-development  Singer  F-16  simulator.  Two 
key  hardware  Interfaces  are  required  to  support 
this  approach.  A  sophisticated  device  Is 
required  to  allow  the  general  purpose  (GP) 
computers  complex  to  act  as  multiple  remote 
terminals  (MRT)  on  the  1553  mux  bus.  This  MRT 
device  allows  the  GP  computer  to  respond 
normally  to  the  FCC  as  the  Inertial  Navigation 
Unit,  Central  Air  Data  Computer,  Radar,  or  any 
other  aircraft  subsystems  that  are  not  present. 
For  example,  the  Stores  Management  Subsystem  was 
not  available  within  the  program  schedule.  The 
complexity  of  the  MRT  Interface  is  due  to  the 
1553  communications  protocol,  primarily  the 
requirement  for  a  subsystem  to  respond  with 
valid  status  within  five  micro-seconds  after  a 
command  transfer  to  that  sub-system.  Since  GP 
computer  software  Intervention  Is  not  possible 
within  that  time  frame,  high  speed  processing 
and  memory  is  required  within  the  Interface. 
(There  are  also  other  stringent  protocol 

requirements.)  The  MRT  device  may  also  serve  as 
a  bus  "monitor"  to  capture  key  Intra-subsystem 
data,  or  as  the  bus  controller.  This  Interface 
did  not  exist  for  the  ASPT  SEL  32/75  GP 
computers.  The  contract  for  this  Interface  was 
let  In  the  first  months  of  the  program.  The 

final  product  was  delivered  about  two  years 
later,  more  than  a  year  behind  firm  schedule 
commitment  despite  technical  relaxation  of 

specifications.  This  development  had  been 

considered  medium  risk,  since  the  subcontractor 

had  developed  a  similar  device  with  less 
sophistication  for  several  other  computers. 

The  other  primary  Interface  required  to 
support  the  FCC  in  a  simulator  system  Is  an 

interface  to  the  FCC's  Automated  Ground  Equip¬ 
ment  (AGE)  port.  This  Interface  allows  GP 

computer  control  such  as  halt  and  run,  program 
loading,  and  rather  slow  speed  Indirect  memory 
access  to  the  FCC  through  a  special  set  of  FCC 
registers.  The  Interface  Is  an  adaption  of  the 
High  Speed  Controller  (HSC)  Interface  of  the 
production  ground  support  mini-computer  system. 

This  Interface  was  constructed  and  tested  via 
contract  within  schedule.  Outstanding  support 

was  provided  by  AFAL,  which  had  previously 
constructed  the  device  for  another  computer. 

The  hardware  "stimulation"  approach  got  more 
involved  when  the  advanced  training/research 

features  had  to  be  considered,  since  the  FCC's 
software  (OFP)  was  not  designed  for  simula¬ 

tion.  Scoring  of  weapons  delivery,  for  example. 
Is  rather  challenging.  One  method  Is  to  access 
the  FCC  memory  locations  for  computed  Impact 
coordinates  through  the  AGF  port  after  detecting 
a  weapon  release  command  on  the  1553  mux  bus  to 
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the  Stores  Management  System.  A  less  Involved 
method  is  to  run  a  ballistics  model  in  the  GP 
Computer  to  score  in  parallel  at  weapon  release 
time,  again  detected  by  bus  data  to  SMS.  The 
implementation  of  a  "Freeze"  function  is  also 
more  than  a  trivial  solution.  One  method  is  to 
utilize  the  bus  monitor  to  store  the  command  and 
data  stream  from  the  FCC.  At  “Freeze"  time,  the 
FCC  is  halted,  the  8CU  interface  is  switched  to 
bus  controller  mode,  and  the  last  list  of  stored 
data  transfers  is  run  continuously.  An  alternate 
method  involves  freezing  the  position  and  atti¬ 
tude  data  from  the  INU  and  accessing  a  few  key 
Integrating  variables  in  the  FCC  through  the  AGE 
port.  These  problems  were  not  solved  on  site  as 
this  approach  was  delayed  more  than  a  year  and  a 
half  by  subcontracted  MRT  interface  schedule 
slipages.  Microcode  problems  in  supporting  the 
complex  multiple  remote  terminal  capability  in 
real  time  were  a  primary  problem.  The  stimula¬ 
tion  approach  was  finally  dropped  to  provide  the 
full  support  to  the  successful  third  approach  as 
on-line  capability  was  demonstrated. 

Emulate  the  FCC  (Translation) 

This  approach  was  based  on  the  translation  of 
the  FCC  software  OFP  from  its  source  language, 
JOVIAL,  into  FORTRAN.  The  FORTRAN  equivalent 
could  then  be  compiled  and  executed  in  the  basic 
flight  simulator  SEL  32/75  GP  Computers.  This 
high  level  language  translation  was  required 
because  a  JOVIAL  compiler  was  not  available  for 
the  SEL  32  computer.  The  hardware  configuration 
to  support  this  approach  is  the  same  as  that  of 
the  simulation  approach;  considerably  simpler 
than  the  stimulation  effort  and  offering  direct 
control  over  the  avionics  system. 

Translation  Implementation 

The  major  effort  in  this  approach  was  the 
massive  translation  effort  as  well  as  the  inte¬ 
gration  of  this  "foreign"  software  Into  the 
simulator  system.  This  approach  was  supported 
through  a  partial  translation  effort.  The 
effort  was  dropped  due  to  the  projected  com¬ 
plexity  of  debugging  and  Integrating  the  massive 
translated  program.  The  complexity  was  due  to 
the  architecture  of  General  Dynamics  Operational 
Flight  Program  (OFP)  which  was  constrained  by 
severe  memory  and  execution  time  limitations. 
The  Integration  problem  was  again  related  to 
this  OFP's  "real-world"  derivations  versus  the 
more  real  time  efficient  simulation  models. 
Insufficient  manpower,  calendar  time,  and  com¬ 
puter  systems  time  for  the  translation.  Integra¬ 
tion,  and  debug  of  this  approach  led  to  this 
approach's  demise  early  in  the  program.  The 
effort  was  greatly  reduced,  then  dropped  to 
provide  full  support  to  the  simulation  design 
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approach  as  it  demonstrated  on-line  capability. 
This  approach  should  not  be  seriously  considered 
unless  the  host  GP  Computer  has  a  JOVIAL  compiler 
or  schedule  and  software  manpower  are  not  consi¬ 
derations. 

Simulate  the  FCC 

The  objective  of  the  FCC  simulation  approach 
was  to  minimize  hardware  and  software  complexity 
and  allow  early  on-line  operation  with  a  priori¬ 
tized  growth  of  the  simulation  envelope.  The 
complexity  of  the  hardware  configuration  to 
support  this  approach  is  drastically  reduced  by 
assuming  the  FCC's  role  of  1553  bus  controller. 
The  configuration  will  be  defined  in  the 
Implementation  section. 

This  design  approach  was  to  accurately  model 
all  FCC  outputs  to  the  hardware-pilot  interface 
subsystems  Implemented  in  Phase  I  of  the  ASPT 
F-16  emulation,  i.e..  Heads  Up  Display  and  Fire 
Control  Navigation  Panel  (FCNP).  FCC  internal 
computations  and  communications  are  eliminated 
wherever  possible.  This  eliminates  the  require¬ 
ment  for  an  Inertial  Navigation  Unit  (INU)  math 
model  and  reduces  the  complexity  of  the  Fire 
Control  Radar  (FCR)  subsystem  model. 

No  malfunctions  or  degraded  modes  are  simu¬ 
lated  except  for  total  subsystem  failures;  i.e., 
FCC,  CADC,  INU  fail.  The  FCC  simulation  approach 
is  of  course  ideally  suited  to  support  the 
various  avionics  failure  modes  when  research  or 
training  requirements  for  this  enhancement  are 
generated. 

The  functional  simulation  of  the  FCC,  there¬ 
fore,  shall  extend  only  to  the  man-machine 
interface.  No  attempt  to  model  the  Internal 
architecture  of  the  operational  flight  program 
(actual  FCC  software)  was  made  due  to  the 
following  reasons: 

-  Weapon  delivery  modes  and  navigation  modes 
are  not  independent  structural  entities  due  to 
memory  limitations.  This  is  not  consistent  with 
the  requirement  for  prioritized  growth  of 
capability  which  can  be  pursued  as  parallel 
development  effort. 

-  OFP  is  generally  not  designed  to  Integrate 
efficiently  with  the  simulation  flight  model. 

-  OFP  uses  re-entrant  routines  extensively. 

-  OFP  is  cluttered  with  failure  testing  and 
degraded  modes. 

Simulation  Implementation 

Hardware 

The  GP  computer  Interface  problems  were 
vastly  reduced  by  assuming  the  FCC’s  role  as  bus 
controller.  The  MRT  device  sophistication  was 
not  required,  and  a  simple  parallel  to  serial 
1553  Interface  (33  Integrated  circuits)  was 
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designed  and  built  In-house.  This  device  was 
mounted  on  a  connector  board  to  an  off  the  shelf 
parallel  interface  (High  Speed  Data  set  -  HSO) 
for  the  SEL  33/75  computer.  All  bus  communica¬ 
tions  are  under  direct  software  control  by  the 
simulator  SP  computers.  The  AGE  port  Interface 
was  no  longer  required,  since  the  FCC  is  not 
used. 

The  aircraft  hardware  subsystems  used  In 
the  simulator  (HUD,  FCNP)  receive  all  digital, 
discrete,  and  analog  signals  normally  presented 
for  any  of  the  simulated  operating  modes  at  data 
rates  greater  than  or  equal  to  that  supported  by 
the  aircraft.  Conmunlcatlons  with  other  simu¬ 
lated  subsystems  such  as  the  SMS  or  FCR  take 
place  through  memory  shared  within  the  same  host 
computer.  The  SMS  subsystem  had  to  be  simulated 
because  a  critical  component  of  that  system,  the 
Central  Interface  Unit  (CIU)  was  not  available 
within  schedule.  The  display  of  the  SMS 
subsystem,  the  Stores  Control  Panel  (SCP)  was 
Interfaced  and  supported  by  the  SEL  32/75  GP 
computer  system.  The  Interface  and  extensive 
Interactive  software  development  was  a 
considerable  effort  and  offered  no  advantages 
over  the  use  of  the  aircraft  hardware. 

Serial  Data  transmission  Rates 
(Hertz) 

Aircraft  Simulation 

50  60 

25  30 

12.5  or  less  15 


Mode,  VIP  Mode,  Snapshoot  Mode  and  LCOS  Mode  have 
been  added  proving  the  ease  of  expansion  Into  the 
air-to-air  simulation  capability. 

The  design  concept  of  allowing  prioritized 
growth  capability  while  maintaining  a  'clean" 
well  structured  system  was  especially  critical 
due  to  the  extremely  compressed  schedule.  The 
development  time  for  each  new  avionics  A/G 
delivery  mode  proved  to  be  greatly  decreased,  as 
each  mode  uses  the  same  avionics  utilities  sub¬ 
routines. 

In  the  area  of  research  support,  the  avionics 
simulation  approach  was  Invaluable  In  Its  ease 
of  interface  with  the  ASPT's  preprograming 
research  system  since  all  "FCC  Internal"  vari¬ 
ables  as  well  as  HUD  Information  were  directly 
accessible  In  the  other  simulator  general 
purpose  computers.  Target  tracking  and  weapons 
scoring,  for  example,  were  directly  available  to 
the  Student  Data  System  (SDS)  and  graphics 
displays  at  the  Advanced  Instructor/  Operator 
Station  through  shared  memory. 

This  approach  was  followed  to  successful 
accomplishment  of  all  design  goals  for  Phase  I 
within  the  300  day  schedule.  More  than  fifty 
F-16  pilots  have  been  trained  (with  research 
data  collected)  in  all  the  visual  air-to-ground 
modes  as  well  as  enroute  navigation,  ILS, 
takeoff  and  landing.  Air-to-air  capability  Is 
being  added  in  parallel  with  visual  system 
updates  to  accommodate  research  requirements  In 
this  area.  Excellent  fidelity  and  training/ 
research  capabilities  have  been  demonstrated. 


Simulation  Modular  Software  Architecture 


Software 

The  desired  software  architecture  was  one 
which  could  easily  support  the  Independent 
development  of  specific  A/G  and  navigation 
modes,  thus  enabling  a  smooth,  prioritized 
growth  of  capability  to  match  research  and 

training  requirements.  A  "top  down"  structural 
approach  within  each  mode  was  applied  to  take 
advantage  of  the  similarities  between  modes; 
each  mode  having  an  executive  program  which 
basically  consists  of  an  organized  series  of 
conmon  utility  subroutine  calls.  Note:  Manual 


CONCLUSION 

All  three  approaches  were  technically  sound 
despite  the  various  difficulties  associated  with 
each.  Each  approach  has  merit  for  a  particular 
simulation  requirement. 

The  hardware  "stimulation"  of  the  FCC,  for 

mb  }Sr,iK"*iisuia«,o»s  sswwsia  » 

on-board  aircraft  avionics  subsystems.  In  a 
training  environment,  however,  there  are  design 
penalties  to  be  considered  as  a  result  of  the 
loss  of  direct  avionics  system  control.  System 
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hardware  complexity  has  a  major  Impact  on 
acquisition  and  life  cycle  costs  as  well  as 
rellablllty/malntalnablllty.  The  difficulties 
In  supporting  conventional  training  features  has 
a  major  Impact  on  software  and  hardware  com¬ 
plexity.  Even  minor  updates  to  the  OFP  which 
represent  little  or  no  operational  change 
require  analysis  and  will  most  likely  cause 
software  changes  due  to  the  relocation  of  key 
variables  (used  for  tracking,  scoring  and 
initializations)  within  the  FCC's  memory. 
(Singer  Co.  has  spent  considerable  effort  in 
developing  software  to  support  this  difficult 
task  for  the  F-16  production  simulator). 

In  a  training  research  or  operational 
research  environment,  this  approach  is  extremely 
cumbersome.  The  software  effort  and  expertise 
required  for  capturing  and  processing  large 
amounts  of  FCC  Internal  Information  or  Imple¬ 
menting  minor  modifications  In  operational 
characteristics  (I.e.,  mode  or  Hud  symbology, 
switchology  or  control  function  changes)  Is 
phenominal.  Post-mortum  research  answers  are 
the  result  of  this  type  of  "hands-tled" 
engineering  situation  where  the  aircraft 
component  (FCC)  is  In  control  rather  than  the 
simulator  GP  computers. 

The  software  "emulation"  of  the  FCC  has 
merit  in  operational  research  and  training/ 
research  only  if  the  host  GP  computer  has  a 
JOVIAL  compiler.  (The  source  high-level 
language  of  the  FCC's  software  OFP).  The 
"emulation"  approach  could  then  possibly  compete 
with  the  hardware  "stimulation"  approach  for  OFP 
software  validation  if  lack  of  hardware  facili¬ 
ties  and  cost  were  primary  factors.  The  positive 
attribute  of  this  approach  for  simulator  applica¬ 
tions  Is  that  the  software  architecture  and 
basic  integrity  of  the  OFP  Is  maintained.  In 
the  "ideal"  world  of  simulation,  however,  many 
of  the  algorithms  may  be  unnecessarily  complex 
and  real-time  consuming.  The  integration  of 
this  "foreign"  software  with  flight  dynamics  and 
advanced  training  software  Is  also  a  major 
software  task.  Detailed  knowledge  of  the  OFP 
and  FCC  hardware  configuration  is  needed  for  an 
efficient  implementation. 


The  "simulation"  system  approach  to  the  FCC/ 
avionics  simulation  affords  the  most  direct  con¬ 
trol,  minimum  hardware  complexity,  and  greatest 
ease  of  Integration  with  the  flight  dynamics  and 
advanced  training  software.  Because  of  these 
positive  features,  the  "simulation"  approach  is 
most  appropriately  applied  to  a  training/research 
or  operations  research  simulation  effort.  In 
these  environments,  the  penalty  of  extended  soft¬ 
ware  development  is  more  than  offset;  particu¬ 
larly  when  priorities  can  be  placed  on  specific 
avionics  system  simulation  capabilities  and 
training  or  research  may  procede  while  software 
development  continues  on  lower  priority  modes  or 
capabilities.  With  this  approach,  critical 
training  or  research  problems  may  be  defined  and 
solved  1-2  years  earlier,  minimizing  early  pro¬ 
gram  aircraft  and  pilot  losses.  Ultimate 
fidelity  is  not  sacrificed  and  system  reliability 
and  maintainability  are  enhanced. 

In  closing,  the  primary  point  to  be  made 
regarding  a  "stimulate"  versus  "simulate"  design 
decision  Is  that  a  good  selection  Is  based  on 
the  specific  projected  requirements,  cost  and 
schedule  for  the  simulation.  Information  and 
reconmendatlons  from  aircraft  hardware  manu¬ 
facturers  or  facilities  whose  requirements  are 
the  evaluation  of  aircraft  hardware  are  extremely 
valuable.  The  program  manager  or  design  engineer 
must  remember,  however,  that  most  of  these 
resources  have  little  understanding  of  the 
simulation  training/research  environment  and  may 
be  fixated  on  the  complexity  of  real-world 
aircraft  system  problems.  If  validation  of 
aircraft  subsystems  Is  not  a  requirement,  a 
simulation  of  that  subsystem  may  be  more 
appropriate;  particularly  when  that  subsystem 
serves  as  a  primary  processor  and  communications 
controller  for  the  entire  avionics  system,  as  In 
the  case  of  the  F-16  Fire  Control  Computer. 
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ABSTRACT 

Engagement  Simulation  (ES)  has  filled  a  training  need  for  maneuver  forces  by  realisti¬ 
cally  assessing  casualties  and  by  replacing  fixed  scenario  exercises  with  free  play  inter¬ 
changes  where  opponents'  actual  behavior  determined  exercise  results.  Artillery  was  not 
integrated  into  ES,  however,  and  the  goal  of  this  research  was  to  develop  a  realistic  and 
inexpensive  ES  method  for  training  artillery  units.  A  computational  system,  which  used  the 
data  actually  set  on  the  guns  and  employed  standard  fire  direction  equipment,  was  designed 
to  select  the  probable  impact  point  of  artillery  rounds.  This  meant  that  the  behavior  of 
artillerymen  directly  affected  the  placement  of  artillery  simulators  on  the  ground.  Thirty- 
six  simulated  missions  were  “fired"  by  an  artillery  battery  during  a  developmental  test. 
Targets  on  the  ground  were  assigned  to  a  forward  observer  and  simulators  were  placed  based 
on  the  computed  impact  points.  Feedback  on  mission  effects  was  provided.  Results  indicated 
that  the  artillery  system  improved  its  speed  and  accuracy. 


INTRODUCTION 

Field  artillery  has  presented  very  specific 
training  problems  throughout  the  history  of  can¬ 
non  gunnery.  These  problems  have  become  more 
acute  in  current  times  because  of  financial  and 
spacial  constraints.  In  years  past,  when  there 
was  a  great  deal  of  open  land  within  the  United 
States,  there  was  generally  no  problem  in  ob¬ 
taining  enough  space  for  live  fire  ranges.  As 
the  country  has  urbanized,  range  limitations 
have  become  severe.  You  need  a  great  deal  of 
land  to  fire  a  live  artillery  piece,  and  safety 
constraints  prevent  fire  into  areas  where  maneuver 
troops  are  training.  The  cost  of  ammunition 
also  has  escalated  dramatically  during  the  past 
decade,  and  this  has  led  to  a  limited  supply 
available  for  training  purposes. 

Due  to  the  limitations  in  training  ammuni¬ 
tion  and  ranges  and  due  to  safety  constraints, 
most  of  the  field  training  within  the  US  Army 
has  been  limited  to  dry  firing  exercises  where 
men  are  required  to  go  through  the  movements  of 
operating  their  weapon  without  any  potential  for 
feedback  as  to  the  success  or  failure  of  their 
fired  mission.  Live  fire  is  restricted  to  sev¬ 
eral  exercises  per  year.  The  learning  literature 
in  psychology  has  emphasized  the  importance  of 
knowledge  of  results  in  order  to  obtain  perfor¬ 
mance  improvement.  You  do  not  receive  this 
knowledge  by  dry  firing  exercises,  by  gun  drills, 
which  involve  just  moving  the  dials  on  the 
weapons,  or  by  displacement  exercises,  where  you 
set  up  the  weapon,  perhaps  dry  fire,  and  then 
move  the  weapon  to  another  location  to  see  how 
fast  you  can  do  it.  In  this  type  of  training 
environment,  the  only  feedback  that  personnel 
receive  is  generally  a  critique  of  what  they  did 
wrong.  There  is  no  possibility  for  positive 
feedback.  Simulation  methods  can  be  very  advan¬ 
tageous  in  a  situation  where  opportunities  for 
"hands  on"  training  through  live  fire  have 
become  increasingly  rare  and  expensive. 


A  system  has  been  evolved  over  the  past 
decade  for  training  combined  arms  (armor  and  in¬ 
fantry)  forces.  Engagement  Simulation  (ES)  is 
the  term  generally  used  to  label  what  amounts  to 
a  family  of  training  systems.  In  the  early  years, 
ES  focused  upon  the  rifle  squad  and  gradually  was 
improved  to  include  the  platoon  and  the  company- 
team  organization.  There  are  several  major 
characteristics  of  Engagement  Simulation  as  it  has 
been  used  for  maneuver  forces.  The  first  is 
weapons  effect  signature  simulation  which  pro¬ 
duced  a  simulated  flash  and  bang  for  each  weapon 
system  similar  to  an  actual  firing.  This  provides 
feedback  to  the  firer  and  cueing  to  his  "enemy" 
which  means  that  the  firer  has  to  take  the  same 
action  that  he  would  if  he  had  fired  at  and 
revealed  his  position  to  an  actual  enemy.  The 
next  and  perhaps  the  most  important  characteristic 
of  ES  is  casualty  assessment.  This  is  a  system 
for  deciding  who  shall  "live"  and  who  shall  "die" 
based  upon  very  precise  rules  which  credibly  re¬ 
present  the  actual  effectiveness  of  soldiers' 
weapons.  Finally,  the  characteristic  which  dis¬ 
tinguishes  ES  from  traditional  field  trainino  is 
what  we  call  the  After  Action  Review.  This  is  a 
participant  interactive  type  of  exercise  which 
occurs  after  the  field  action  and  amounts  to 
facilitating  the  communication  between  the  two 
opposition  forces.  They  can  review  the  action  and 
determine  the  effective  ami  ineffective  behaviors 
in  a  noncritical  setting. 

When  ES  was  first  developed,  a  method  was 
required  to  simulate  artillery  to  the  extent 
necessary  so  that  ground  maneuver  forces  would 
know  that  it  existed.  The  focus  was  on  artillery 
from  the  viewpoint  of  the  infantryman  rather  than 
from  the  viewpoint  of  the  artilleryman.  The 
method  employed  for  maneuver  forces  was  both 
basic  and  straightforward.  An  Infantry  commander, 
for  example,  would  ask  for  artillery  fire  at  a 
particular  location;  a  fire  marker  control  center 
(FWCC)  would  plot  the  location  on  a  map  and 
provide  directions  to  individual  fire  markers. 


They  were  told  to  go  to  the  requested  location 
and  drop  artillery  simulators  on  the  ground.  The 
location  to  which  they  were  sent  was  the  same  as 
the  location  where  fire  was  requested.  This  was 
very  unrealistic  because  it  eliminated  all  effects 
of  artillery  processing  time  and  the  possibilities 
for  error.  These  can  be  considerable,  especially 
on  the  first  round  fired.  This  method  did  not 
involve  any  artillerymen  except  the  forward 
observer,  who  if  he  was  assigned  to  the  maneuver 
commander,  could  request  the  fire  missions. 

The  research  problem  was  to  develop  a  method 
to  incorporate  the  field  artillery  firing  system 
into  an  Engagement  Simulation  training  program. 

The  Field  Artillery  System  consists  of  three 
main  elements.  As  previously  noted,  the  forward 
observer  is  located  with  the  maneuver  units  and 
has  the  task  of  calling  for  fire  missions  based 
upon  the  needs  of  the  maneuver  unit  commander. 

He  sends  these  calls  to  the  Fire  Direction  Center 
(FDC),  where  computations  are  accomplished  and  the 
data  is  sent  to  the  final  step  in  the  firing 
system,  the  gun  battery.  The  gun  battery  sets 
the  data  on  their  guns  and  ammunition  and  fires 
the  mission.  The  forward  observer  observes  the 
incoming  fire,  makes  his  adjustments  and  sends 
this  back  through  the  system.  This  cycle  is 
repeated  until  the  rounds  hit  the  target.  In 
traditional  ES,  the  FDC  and  the  gun  battery 
itself  were  not  involved  in  any  way. 

The  methodological  problem  generated  by  this 
research  effort  was  how  to  get  the  gun  battery 


and  the  FDC  involved.  A  concept  for  solution  was 
to  develop  a  method  of  determining  the  terminal 
effects  of  the  simulated  round  (the  projectile 
fired  from  artillery),  specifically  to  locate  the 
most  probable  impact  point  of  that  round.  This 
system  had  to  be  as  realistic  as  possible,  in¬ 
expensive,  and  implementable  on  the  limited  range 
space  where  units  train  most  of  the  time  at  home 
station.  Weapon  signature  simulation,  which  is 
part  of  maneuver  ES,  was  not  required  for  the 
purposes  of  involving  the  FDC  or  the  qun  battery. 
What  we  were  most  interested  in  was  integrating 
the  performance  of  people  who  are  directly  in¬ 
volved  in  the  artillery  firing  sequence,  so  that 
feedback  could  be  provided  on  the  accuracy  and 
speed  of  their  performance. 

METHOD 

A  standard  artillery  procedure  was  already 
available  to  provide  the  basic  solution  to  our 
problem.  This  procedure  is  called  replot  and  is 
used  to  determine  the  location  of  a  target  from 
the  data  (settings  for  the  guns)  that  were  com¬ 
puted  by  the  FDC  and  sent  to  the  gun  battery. 
However,  as  used  in  standard  artillery  practice, 
replot  is  a  relatively  time  consuming,  iterative 
procedure.  It  was  necessary  to  simplify  this  pro 
cedure  and  to  acquire  the  data  directly  from  the 
gun  battery  in  an  accurate  and  timely  fashion. 
These  data  had  to  be  sent  to  a  control  center,  be 
replotted,  and  then  be  sent  further  forward  to 
fire  markers.  They  could  then  mark  the  target, 
not  where  fire  was  requested,  but  where  the  fire 
would  have  gone  given  all  the  inputs  of  the 
artillery  system. 


FORWARD  OBSERVER 


FIGURE  1 

THE  ARTJUKY  ENGAGEMENT 
SIMULATION  SYSTEM 


Figure  1.  Artillery  Engagement  Simulation  System 


291 


The  system  that  was  developed  is  described  in 
Figure  1.  Each  element  of  the  artillery  firing 
system  has  a  corresponding  element  in  an  ES  con¬ 
trol  system.  The  role  of  the  fire  marker,  who  is 
in  the  target  area,  is  the  same  as  it  was  in  the 
traditional  ES  except  that  he  is  controlled  by  a 
fire  marker  control  center  (FMCC)  which  gives  him 
directions  based  upon  computations  which  will  lead 
him  to  the  most  probable  impact  point.  Fire 
markers  are  pre-positioned  on  surveyed  points  and 
given  a  direction  and  a  distance  which  they  can 
pace  off  or  drive.  They  then  mark  their  target 
using  "flash-bang"  simulators  and  retire  to  the 
closest  surveyed  point.  The  work  of  the  fire 
marker  essentially  represents  the  terminal  effects 
of  the  control  system. 

Prior  to  the  fire  marker  ever  getting  any 
instruction,  the  system  is  driven  by  what  happens 
in  the  gun  battery.  The  forward  observer  calls 
his  mission  to  the  fire  direction  center  (FDC); 
the  FDC  does  its  computations  and  sends  the  data 
to  the  gun  battery;  crews  set  the  data  on  the  guns 
and  dry  fire  their  weapons.  At  that  point,  con¬ 
trol  personnel,  who  remain  out  of  the  way  of  the 
gun  crews,  must  examine  the  gun  sights  and  the 
ammunition  very  quickly,  obtain  the  data  elements, 
and  send  this  data  to  the  fire  marker  control 
center  (FMCC).  The  FMCC  performs  replot  in  two 
ways.  First,  they  use  a  tradional  plotting  board 
approach,  which  has  been  available  for  years  but 
never  used  in  this  manner  for  ES,  and  as  previous¬ 
ly  indicated  is  very  time  consuming.  Also,  they 
use  a  TI59  calculator  with  a  special  program  that 
was  developed  in  this  research  project  to  perform 
replot.  The  reason  for  doing  it  two  separate  ways 
was  to  provide  cross-checks  for  accuracy  and  to 
evaluate  the  two  methods.  When  the  two  plots 
agree,  the  data  is  sent  to  the  fire  markers. 

When  the  two  plots  disagree,  it  is  necessary  to 
find  and  resolve  the  error  prior  to  transmitting 
the  data.  Once  the  targets  are  marked,  the  job 
of  the  forward  observer,  part  of  the  artillery 
firing  system,  is  to  correct  his  initial  call  for 
fire.  He  therefore  provides  feedback  (especially 
if  he  thinks  he  is  not  getting  what  he  has  asked 
for)  on  a  continuing  basis  to  the  battery  through 
the  FDC  so  that  a  performance  feedback  loop  exists. 


Developmental  Tests 


In  the  fall  of  1979,  a  developmental  test  of 
artillery  engagement  simulation  was  accomplished. 
Participants  in  the  test  included  an  artillery 
firing  battery,  from  a  direct  support  artillery 
battalion,  two  sets  of  forward  observers,  an  FDC, 
and  enough  personnel  drawn  out  of  the  battalion 
to  provide  for  gun  controllers,  an  FMCC,  and  fire 
markers.  Personnel  in  the  control  system  were 
selected  because  their  control  tasks  were  similar 
to  their  regular  military  duties.  They  were 
trained  in  these  tasks  during  a  three-day  period. 
All  equipment  in  the  developmental  test  was  equip¬ 
ment  which  is  currently  standard  within  a  field 
artillery  battalion.  Fire  markers  were  issued 
"flash-bang"  simulators  which  they  used  to  mark 
the  targets  on  the  ground,  and  communications 
were  set  up  using  standard  tactical  radios  and 
telephones  for  the  control  system  so  that  the 
FMCC  could  transmit  probable  impact  points  to  the 
fire  markers,  and  could  receive  the  gun  data  from 
the  controllers  in  the  gun  battery. 


A  tactical  setting  was  written  which  in¬ 
cluded  scenarios  designed  to  provide  a  represent¬ 
ative  array  of  tasks  involved  in  operating  a  dir¬ 
ect  support  artillery  battery.  There  were  no 
maneuver  troops  available;  a  researcher,  who  had 
served  as  an  Artillery  Officer,  role  played  as 
the  maneuver  company  commander.  He  moved  with 
the  forward  observer  teams,  and  designated  tar¬ 
gets  during  attack  and  defense  missions.  During 
the  developmental  tests,  36  missions  were  fired. 
Each  mission  consists  of  an  initial  call  for  fire 
plus  all  adjustments.  This  led  to  82  separate 
firings  of  the  battery  over  approximately  a  three- 
day  period.  All  missions  were  during  daylight 
operations  in  relatively  clear  weather. 

The  procedure  for  obtaining  gun  data  from 
the  battery  was  accomplished  by  randomly  checking 
one  of  the  six  guns  in  the  battery,  and  data  were 
sent  forward  to  the  FMCC.  Every  gun  crew  was  made 
aware  of  the  fact  that  its  performance  could 
potentially  affect  the  results  of  the  entire 
battery.  Gun  controllers  were  instructed  to  stay 
out  of  the  way  of  the  gun  crew  and  to  establish 
a  professional  relationship  with  the  crew  chiefs, 
such  that  the  data  that  was  sent  forward  had 
credibility  for  the  crews,  i.e.,  if  the  crew  did 
not  respect  the  gun  controller's  professional 
ability,  the  system  would  not  have  had  any  face 
validity. 

RESULTS 

The  performance  of  the  control  system  was 
very  important  in  terms  of  evaluating  the  overall 
training  program.  The  time  data  were  collected 
using  electronic  stopwatches  from  the  initial 
call  for  fire  through  final  marking  of  targets. 
Complete  data  were  collected  on  24  missions  out 
of  the  total  36.  Table  1  describes  the  respon¬ 
siveness  of  the  control  system  over  those  2 A 
missions  by  breaking  the  missions  down  into  four 
blocks  of  six  missions  each  and  computing  the 
median  for  each  mission.  Gun  controller  delays 
appear  to  be  minimal  from  the  first  series  of 
missions,  and  do  not  appear  to  change  appreciably 
over  the  four  mission  blocks.  The  fire  marker 
control  center  does  improve  somewhat  over  the 
four  blocks.  However,  the  biggest  improvement 
can  be  seen  with  the  fire  markers  who  begin  with 
an  average  response  time  of  over  three  minutes 
and  get  their  response  time  down  to  almost  two 
minutes.  By  the  fourth  mission  block,  it  can  be 
seen  that  the  median  delay  was  slightly  over 
three  minutes  for  the  control  system.  The 
median  increase  in  the  third  block  was  primarily 
a  function  of  fire  marker  performance,  which  may 
have  been  related  to  the  trafficability  of  the 
terrain. 

Table  1 


MEDIAN  CONTROL  SYSTEM  RESPONSIVENESS 
(SECONDS) 


Mission 

Block 

Gun 

Controllers 

FMCC 

Fire 

Markers 

Overal 

I 

31.0 

108 

216.5 

377 

II 

28.0 

86 

177.0 

291 

III 

31.10 

87.5 

247.0 

354 

IV 

27.5 

67.0 

129.5 

208.5 

292 


The  performance  of  the  fire  marker  was  re¬ 
lated  to  the  distance  that  he  had  to  travel.  A 
linear  regression  analysis  provided  a  regression 
equation:  t=84.07  +  0.87d.  The  Pearson  corre¬ 
lation  between  distance  and  time  was  0.73,  which 
was  significant  (p<.01). 

During  the  82  firings  of  the  exercise,  the 
FMCC  transmitted  only  one  error  to  the  fire 
markers.  Five  other  errors  were  corrected  in 
cross-checks  by  the  two  computers,  the  one  using 
the  board  and  the  other  using  the  calculator  in 
the  FMCC.  It  was  found  that  the  calculator  method 
was  more  accurate  in  computing  the  most  probable 
impact  point,  because  it  was  a  precise  mathemati¬ 
cal  procedure.  The  board  plotting  involves  cer¬ 
tain  perceptual  limitations  of  the  operator.  Fire 
marker  accuracy  was  analyzed  for  the  first  time  in 
this  exercise.  There  is  no  way  of  knowing  pre¬ 
cisely  whether  or  not  it  is  comparable  to  previous 
ES  exercises.  The  mean  cumulative  mission  error 
for  fire  markers  was  about  75  meters.  The  mean 
error  per  shot  was  somewhat  less  than  that,  about 
48  meters.  This  is  within  the  burst  radius  of  the 
type  of  artillery  employed,  155mm  howitzers,  and 
therefore  is  not  considered  extreme. 

The  critical  test  of  any  training  system  is 
how  it  affects  the  behavior  of  the  individuals  or 
organization  being  trained.  The  most  important 
aspects  of  artillery  system  performance  are  speed 
of  delivery  and  accuracy.  Data  were  collected  on 
these  variables  for  36  missions. 
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ARTILLERY  SYSTEM  DELAY 


Figure  2.  Artillery  System  Delay 

The  36  missions  fired  during  the  exercise 
were  divided  into  three  blocks  of  12  missions, 
and  the  mean  delay  for  a  call  for  fire  until  the 
first  round  was  delivered  on  target  by  the  artil¬ 
lery  system  was  computed.  These  delays  did  not 
Include  the  control  system  time  which  was  sub¬ 
tracted  from  the  overall  delay.  Figure  2  de¬ 


scribes  the  change  in  system  performance  from  the 
first  through  the  third  mission  block.  As  can  be 
seen,  there  is  a  considerable  improvement.  The 
system  gets  faster  as  training  progresses. 

The  accuracy  of  the  artillery  firing  system 
was  determined  by  the  distance  between  the  co¬ 
ordinates  requested  by  the  forward  observer,  who 
is  part  of  the  system,  and  the  impact  coordinates 
computed  by  the  FMCC.  Again,  any  error  produced 
by  the  control  system  was  not  included  in  these 
computations. 


Figure  3.  Artillery  System  Accuracy 


Figure  3  describes  the  change  in  accuracy 
from  the  first  mission  block  of  12  missions 
through  the  third  mission  block.  As  can  be  seen, 
the  greatest  improvement  occurred  between  the 
first  and  second  mission  blocks.  By  the  third 
mission  block,  accuracy  was  well  within  the  burst 
radius  of  one  round  of  artillery,  which  means  that 
the  unit  was  performing  in  a  very  creditable 
fashion. 

The  training  system  will  not  be  successful 
if  the  trainees  resist  the  training.  An  evalua¬ 
tion  of  the  artillery  engagement  simulation  system 
would  not  be  complete  without  some  input  by  the 
participants.  Figure  4  describes  some  selected 
items  of  information  that  were  requested  from  the 
artillerymen,  who  participated  in  this  exercise. 
They  were  asked  to  respond  on  an  eight-point 
scale  from  strong  disagreement  to  strong  agreement 
concerning  some  statements  about  their  training. 

An  average  or  mean  response  higher  than  midpoint 
of  the  scale  indicates  some  level  of  agreement  and 
those  below  the  midpoint  indicate  some  level  of 
disagreement.  As  seen  in  Figure  4,  the  gun  con¬ 
trollers  had  credibility  with  the  crews,  and  on 
the  average  crews  felt  that  the  Information  about 
their  performance  made  the  training  more  interest¬ 
ing.  They  also  indicated  that  the  controllers  did 
not  get  in  their  way  while  they  were  doing  their 
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THE  CONTROLLER  REPORTING  OF  GUN  DATA  WAS 
ACCURATE 


THE  INFORMATION  ON  BATTERY  ACCURACY  MADE 
TRAINING  MORE  INTERESTING 


THE  CONTROLLERS  DID  NOT  INTERFERE  WITH  THE 
PBIFORMANCE  OF  YOUR  JOB 


MV  MOST  RECENT  FIELD  TRAINING  WAS  BORING 


DRY  FIRING  EXERCISES  ARE  ALWAYS  A  WASTE  OF 
TINE 


*  1  2  3  4  5  6  7  8 

STRONG  RESPONSE  SCALE  STRONG 
DISAGREEMENT  AGREEMENT 

FIGURE  4 

MEAN  PARTICIPANT  RESPONSE 


Figure  4.  Mean  Participant  Response 

jobs.  Two  scales  that  were  introduced  to  see 
whether  or  not  negative  responses  were  possible 
indicated  that  while  people  did  not  disagree 


strongly,  they  were  at  least  ambivalent  to  state¬ 
ments  such  as  "my  most  recent  field  training  was 
boring,"  and  "dry  firing  exercises  are  always  a 
waste  of  time."  Informal  coirments  by  forward  ob¬ 
servers  indicated  that  they  considered  the  accu¬ 
racy  and  speed  of  the  artillery  engagement  simula¬ 
tion  system  comparable  to  live  fire,  because  live 
fire  introduces  many  delays  due  to  safety  con¬ 
straints.  They  liked  the  training  method  better 
than  other  non-firing  methods  such  as  the  sub¬ 
caliber  mortar  range,  which  is  very  responsive  to 
weather  and  wind  conditions. 

CONCLUSIONS 

While  this  method  for  integrating  artillery 
ES  into  a  larger  simulation  training  system  has 
not  been  attempted  with  maneuver  troops  due  to 
budgetary  constraints,  it  is  believed  that  the 
method  is  functional  and  will  serve  to  add  real¬ 
ism,  both  for  artillerymen  and  for  maneuver  com¬ 
manders.  The  beauty  of  this  method  is  that  it 
employs  no  additional  equipment  other  than  what  is 
already  in  an  artillery  battalion,  and  requires 
few  personnel  to  run  it.  There's  still  no  tech¬ 
nological  solution  for  marking  targets  other  than 
the  one  that  has  been  employed  over  the  last  ten 
years,  having  a  man  drop  a  simulator  on  the 
ground.  Fire  markers  must  be  carefully  trained 
or  they  won’t  perform  in  an  accurate  and  timely 
fashion.  Developmental  tests  have  led  to  the 
point  where  there  is  now  an  ES  training  program 
that  involves  actual  artillerymen  on  the  ground 
using  their  standard  equipment.  This  is  a  big 
step  forward.  Hopefully,  in  years  to  come,  a 
validational  effort  can  be  mounted  in  conjunction 
with  maneuver  troop  exercises. 
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THE  SEMI-AUTOMATIC  INSTRUCTIONAL  SYSTEM 


William  S.  Reese  &  Robert  A.  Hansen 
Link  Division  of  The  Singer  Company 
Binghamton,  New  York 
ABSTRACT 

This  paper  outlines  Link  division's  approach  to  the  general  training  requirements  of  the  B-52  WST, 
and  the  specific  requirements  of  the  Instructional  System.  A  review  of  the  analysis  and  design  approach 
is  presented,  as  well  as  an  overview  of  the  resulting  Instructional  System,  including  several  of  the 
factors  which  influenced  the  design  and  development  of  the  system.  Several  design  goals  were  established 
for  the  Instructional  System  and,  at  this  writing,  the  system  has  undergone  several  months  of  limited 
qualification  testing  by  the  Air  Force  test  team,  culminating  in  a  production  contract  award  to  Link. 

INTRODUCTION 


As  the  developmental  phase  of  the  B-52  Weapon 
System  Trainer  (WST)  program  nears  a  close,  a 
training  device  has  emerged  which  in  many  ways 
reflects  the  dynamic  environment  of  its  airborne 
counterpart.  This  paper  is  aimed  at  helping  the 
audience  understand  the  evolution  of  this  system. 

We  will  first  present  a  brief  overview  of  the 
aircraft,  and  then  look  at  some  general  require¬ 
ments  for  the  WST  as  well  as  more  specific  In¬ 
structional  System  requirements.  The  next  step  is 
to  describe  the  resulting  instructional  system 
which  was  developed  to  meet  those  requirements 
along  with  several  other  influencing  factors. 

In  a  program  of  this  complexity  and  duration, 
one  expects  a  certain  evolution  of  design  goals 
and  system  requirements  based  upon  new  data  and 
observed  system  utilization  by  line  crewmembers. 
This  was  certainly  the  case  during  the  development 
of  the  B-52  WST.  The  additional  challenges  and 
frustrations  of  the  competitive  procurement  also 
added  to  the  general  complexity  of  the  effort. 
Contributing  to  this  is  the  complexity  of  the  B-52 
equipment,  requiring  six  crewmembers  (Pilot, 
Copilot,  Navigator,  Radar  Navigator,  Electronic 
Warfare  Officer,  and  Gunner). 

Classified  as  a  heavy  strategic  bomoer,  its 
primary  mission  is  the  delivery  of  strategic 
weapons  wherever  such  delivery  is  deemed  appro¬ 
priate.  The  physical  layout  of  the  aircraft  seg¬ 
ments  the  crew  as  shown  in  Figure  1.  The  Pilot  and 
Copilot  are  positioned  in  the  forward  area  of  the 
upper  deck,  and  are  responsible  for  the  basic 
flight  control  of  the  aircraft.  The  Pilot,  *<ho 
occupies  the  left  seat,  is  also  the  Aircraft 
Commander,  responsible  for  the  overall  mission  and 
crew.  The  aft  area  of  the  upper  deck  is  occupied 
by  a  defensive  team  consisting  of  Electronic 

Warfare  Officer  and  Gunner,  responsible  for  the 
defense  of  the  ship  from  ground  and  air-based 
threats  such  as  anti-aircraft  artillary,  surface- 
to-air  missiles,  air  Interceptors,  and  air- 

launched  missiles.  The  lower  deck  is  occupied  by 
an  offensive  team,  a  Radar  Navigator  and  Naviga¬ 
tor,  responsible  for  the  accurate  navigation  and 
direction  of  the  aircraft  from  takeoff  to  landing 
and  the  direction  of  the  aircraft  for  release  of 
gravity  weapons.  The  offensive  team  Is  also 

responsible  for  the  Short  Range  Attack  Missiles 
carried  aboard  the  B-52  for  use  against  ground- 
based  threats  in  an  attempt  to  Improve  the 

penetrating  B-52's  probability  of  survival. 


Figure  1  B-52  CREW  POSITIONS 


The  B-52's  mission,  though  simply  stated  (de¬ 
liver  weapons  on  a  designated  target),  becomes 
complicated  due  to  the  performance  of  the  aircraft 
and  systems  while  operating  in  an  external  envi¬ 
ronment  of  higher  (state-of-the-art)  technology 
and  additionally  due  to  the  intricate  coordination 
of  activities  required  of  the  crew  members. 

The  prime  item  development  specification  for 
the  WST  requires  the  delivery  of  three  separate, 
but  joinable,  training  devices.  A  Flight  Station 
with  six-degree-of-freedom  (6-DOF)  motion  and 
digital  visual  system,  an  Offensive  Station  writh 
3-DOF  motion  and  digital  radar  landmass  system, 
and  a  Defensive  Station  with  an  interactive  threat 
environment.  Each  station  additionally  Includes  a 
remote  Instructor  Station,  and  a  secondary  on¬ 
board  Instructor  Station  and,  of  course,  a  com¬ 
putational  system.  The  physical  layout  of  the 
aircraft,  shown  In  Figure  2,  makes  the  separation 
of  the  crew  Into  three  units  In  a  trainer  quite 
attractive.  Figure  3  Is  a  schematic  representation 
of  the  three  crew  stations  which  make  up  the  6-52 
WST.  Since  members  of  the  three  teams  rarely  make 
direct  visual  contact  writh  a  member  of  another 
team,  the  situation  In  the  trainer  Is  not  signifi¬ 
cantly  different  from  that  in  the  actual  aircraft. 


f  LIGHT  TEAK 


7.  Instructional  Communication  System  (ICS). 


Instructional  System  Requirements/Analysis 

The  prime  item  development  specification  re¬ 
quired  a  system  to  meet  training  needs  for  initial 
combat  crew  qualification  and  for  mission  quali¬ 
fication  and  continuing  training  and  to  accommo¬ 
date  operation  for  instruction,  proficiency  train¬ 
ing  and  evaluation. 

Specific  capabilities  and  features  included: 

1.  Remote  Instructor  Stations  for  each  crew 
station  --  Flight  (FIS),  Offensive  (OIS),  Defen¬ 
sive  (DIS)  —  with  each  optimized  for  dedicated 
operations  with  its  associated  student  station. 

2.  Graphic  CRT  display/function  keyboard 
system  to  provide  for  Programnlng  (setup  and 
modification  of  training  mission  data);  Status  of 
cockpit  equipment,  student  actions,  and  the 
training  situation;  and  Plots  of  aircraft  track 
during  various  mission  phases. 

3.  Simulator  controls  for  all  functions 
necessary  for  training. 

4.  Repeater  display  for  selected  equipment, 
including  visual  system,  Electro-optical  Viewing 
System  (EVS)  monitor,  and  radar  Indicator,  as  well 
as  selected  Defensive  Station  repeater  displays. 

5.  On-board  instructor  positions  to  allow 
over-the-shoulder  instruction  and  evaluation. 

6.  Simulation  Exercise  Control  Unit  (SECU) 
to  provide  the  on-board  Instructor  with  limited 
control  over  the  simulation  exercise. 


8.  Automatic  Monitoring  of  simulation  vari¬ 
ables  and  parameters,  flight  profiles  and  pro¬ 
cedures  with  hardcopy  printout,  CRT  display,  and 
magnetic  disk  storage  of  monitored  data  for  post- 
mission  analysis. 

9.  Automatic  Playback  of  simulator  perfor¬ 
mance. 

10.  An  automatic  radio  message  system. 

11.  Automatic  malfunction  insertion. 

12.  Capability  to  record  and  reset  to  simula¬ 
tor  position  and  conditions  during  real-time 
operation. 

13.  Freeze/Unfreeze  simulated  station  as  well 
as  selected  parameters. 

14.  Hardcopy  printout  of  mission  file  data 
and  other  data  pertinent  to  instruction. 

15.  Crewmember  performance  scoring. 

Added  to  these  requirements  is  an  additional 
requirement  for  analysis  of  the  instructional 
roles  to  be  filled  by  the  WST,  leading  to  the 
definition  of  the  basic  system  structure  and 
specific  methods  to  be  used.  These  specific 
details  as  well  as  overall  goals  of  the  instruc¬ 
tional  system  were  defined  based  upon  the  results 
of  this  process.  A  primary  factor  in  goal  defini¬ 
tion  is  the  analysis  of  the  skills  and  activities 
to  be  rehearsed  in  the  trainer  and  the  levels  of 
proficiency  of  those  who  will  be  trained.  The 
requirements  of  the  B-52  specification  provide  a 
collection  of  features  and  capabilities  *rf)1ch  are 
meant  to  support  these  activities  and  partici¬ 
pants,  but  in  order  to  properly  and  effectively 
implement  a  system,  further  study  was  required. 

In  the  B-52,  the  primary  mission  goal  and 
many  of  the  secondary  or  enabling  goals  are  not 
rehearsable  in  the  aircraft.  This  is  apart  from 
the  normal  emergency  procedure  drills  and  equip¬ 
ment  failure  problems  which  are  considered  too 
risky  in  any  aircraft.  The  analysis  of  instruction 
system  needs  must  be  grounded  primarily  on  the 
needs  of  the  crewmembers  in  retaining  a  high  level 
of  proficiency  in  this  complex  environment. 

Further,  the  instructional  system  designed 
for  the  WST  requires  a  significant  degree  of 
tactical  flexibility.  Not  only  do  the  methods  of 
modern  warfare  change  frequently,  but  the  basic 
tools  also  are  subject  to  development.  The  promise 
of  the  cruise  missile  and  state-of-the-art  avion¬ 
ics  systems  hold  the  possibility  of  vast  changes 
in  the  role  of  the  B-52  crewmember.  Since  the 
effects  of  these  new  developments  on  the  B-52‘s 
detailed  training  requirements  are  at  present 
certain,  the  basic  system  developed  for  the  WST 
had  to  respond  effectively  and  easily  at  a  moder¬ 
ate  cost  to  fairly  significant  changes  in  the 
employment  role  of  the  B-52  Weapon  System. 

For  those  not  familiar  with  details  of  the 
B-52  WST  procurement,  it  was  administered  by  the 
United  States  Air  Force  as  a  competitive  flyoff 
between  the  Boeing-Wi chlta  Co.  and  Singer-Link 
Division,  both  of  whom  designed  and  built  protoype 
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WST's.  These  trainers  were  then  both  subjected  to 
testing  by  Air  Force  personnel,  culminating  In  a 
final  flyoff  conducted  by  actual  SAC  line  crew¬ 
members.  Throughout  the  development.  Air  Force 
guidance  was  severely  limited  so  as  to  avoid 
giving  an  edge  to  one  competitor  over  another. 

As  a  result,  very  little  official  guidance  or 
feedback  on  proposed  modifications  and  deviations 
could  be  provided,  and  areas  of  the  specification 
which  were  goal-oriented  could  be  interpreted  In 
ways  which  our  analysis  indicated  would  provide 
the  most  effective  system.  Other  areas  which  were 
very  specific  not  only  in  the  goal,  but  also  in 
specific  Implementation  techniques,  were  more 
troublesome,  since  often  more  effective  means  were 
available.  One  such  area  was  in  simulation  of  an 
air  refueling  rendezvous,  where  a  specific  model 
was  described,  although  alternative  mthods  not 
only  simplified  the  problem,  but  in  fact  resulted 
in  a  more  faithful  reproduction  of  real-world 
events.  In  the  competitive  environment,  there  is 
considerable  pressure  to  conform  to  specification 
dictates  precisely,  so  alternative  approaches  were 
examined  closely  before  acceptance  or  rejection. 

The  specification,  as  a  primary  design  source 
document,  provided  a  wealth  of  information  on  the 
requirements  of  the  system.  To  add  to  this  source, 
a  number  of  other  data  sources  were  combined  to 
provide  an  overall  picture  of  the  instructional 
needs  of  B-52  users.  Sources  such  as  Instructional 
System  Development  (ISD)  Analysis  performed  for 
the  Strategic  Air  Command  in  1974  provided  valu¬ 
able  guidance  in  evaluating  typical  task  elements 
and  problem  areas  in  the  training  programs  then  in 
existence  for  new  B-52  crewmembers.  Other  military 
sources,  such  as  the  Education  Training  Require¬ 
ments  { ETR )  document,  further  enumerated  indivi¬ 
dual  task  elements  for  the  operational  crewmember. 

The  conclusion  reached  through  this  process 
contrasts  the  B-52  WST  with  other  multistation 
trainers.  Although  the  three  stations  which 
comprise  the  B-52  WST  are  designed  for  use  In 
independent  as  well  as  integrated  training,  a 
major  factor  in  their  design  is  the  interrela¬ 
tionship  of  tasks  among  the  crew  members.  In 
separated  training  situations,  the  instructor  is 
burdened  with  the  responsibility  for  role-playing 
to  provide  the  required  assistance  or  hindrance 
which  would  normally  cane  fran  other  crewmembers. 
As  the  time  comes  to  Integrate  Individual  skills 
into  a  mission-objective-oriented  skill  package. 


the  interplay  among  the  crew  becomes  a  major 
factor. 

To  fine-tune  the  mission-critical  skills,  the 
true  complexity  of  the  mission  environment  must  be 
allowed  to  unfold.  The  situation  here  is  sharply 
different  from  that  existing  In  a  multi-student 
system  trainer,  where  a  number  of  students  are 
performing  their  missions  in  a  completely  un¬ 
related  way.  The  coordination  tasks  for  the 
Instructors  in  an  integrated  WST  are  considerable 
and  require  direct  support  in  the  design  of  the 
machine. 

Another  major  requirement  exists  for  rapid 
and  direct  transition  amoung  the  available  oper¬ 
ating  modes  of  the  WST.  This  transition  assumes 
key  Importance  with  the  realization  that  WST  time 
is  utilized  best  by  developing  training  scenarios 
which  exploit  mode  transitions  to  bring  the  train¬ 
ees  together  for  complex  mission  elements  while 
allowing  Independent  pursuit  of  relatively  non- 
coordinated  tasks. 

In  developing  the  design  goals  for  the  WST, 
the  primary  goals  were  based  upon  the  complex  task 
structure  in  the  trainees'  required  skill  inven¬ 
tory,  and  the  complex  environment  in  which  the 
crew  must  be  placed.  Based  upon  a  consideration  of 
these  factors,  many  secondary  or  derived  require¬ 
ments  emerged  which  were  significant  to  the  imple¬ 
mentation  techniques  used  for  the  Instructional 
System  software.  A  good  example  of  this  is  related 
to  the  previous  discussion  of  trainer  mode 
control . 

What  emerges  from  a  study  of  this  problem  is 
a  group  of  central  design  considerations.  These 
are  the  fundamental  goals  which  all  secondary 
goals  and  specific  system  requirements  are  built 
upon.  Table  1  reviews  some  central  elements  of  the 
WST  goal/approach  structure.  Due  to  the  highly 
complex  task  structure  which  the  WST  instructor 
must  monitor,  his  role  as  an  evaluator  and 
diagnostician  must  be  optimized.  The  required 
simulation  injects  the  simulated  aircraft  into  a 
fairly  complex  environment.  To  the  greatest  extent 
possible,  the  environment  must  be  controlled  auto¬ 
matically,  with  status  clearly  and  easily  reported 
to  the  instructor. 

Finally,  since  the  mission  of  the  B-52  and 
the  interrelationship  of  tasks  for  the  B-52  crew 
is  so  complex,  an  easy  method  must  be  available 


TABLE  1  DESIGN  GOALS/ APPROACH  SUMMARY 


SIM>LIFY  OPERATION 

REDUCE  INSTRUCTOR  TASK  LOADING 

OPTIMIZE  INSTRUCTIONAL  MEDIA 

0  SINGLE  KEY  PAGE  SELECTION 

0  MISSION  SEQUENCE/EVENTS 

0  CONTROL  PANEL  CONFIGURATION 

0  COLOR-COOED  KEY  FUNCTIONS 

0  MALFUNCTION  ACTIVATION  AND  OELECTION 

0  CRT  DISPLAY  ORGANIZATION  AND  CONTENT 

0  USE  OF  SPECIAL  FUNCTION  KEYS 

0  PROCEDURE  MONITORING 

0  LOGICAL  CONTROL  INTERACTIONS 

STANDARDIZE  INSTRUCTION 

0  PRE-SPEC I F IEO  LIBRARY  MISSIONS 

0  PRE-SPECIFIED  INSTRUCTIONAL  DATA 

0  PERFORMANCE  STATUS  AND  MONITORING 

0  PROCEDURE  MONITORING 

0  HARDCOPY  PRINTOUT 

0  CONFIGURATION  CONTROL 

0  VOICE  MESSAGES 

0  PERFORMANCE  MONITORING  AND  RECORDING 

0  DISPLAY  UPDATING 

0  CONTROL  FUNCTIONS 

SIM>LIFY  DATA  GENERATION  PROCEDURES 

0  STANDARDIZED  INPUT  FORMATS 

0  PRE-FORMATTED  DATA  INPUT  SHEETS 

0  DISPLAY  PAGE/DATA  CORRELATION 

0  DATA  VALIDATION 

0  AUTOMATION  OF  CONTROL  INPUTS 
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whereby  careful  planning  of  a  training  session  may 
be  performed  prior  to  the  training  period,  easily 
recalled  for  use,  and  executed  in  a 
straightforward  manner.  Improvisation  is  tolerated 
by  the  system  to  allow  for  interactive  tailoring 
of  the  training  situation  based  on  trainee 
performance,  but  the  key  design  factor  is  the 
persistent  notion  that  the  interdependence  of  the 
elements  in  the  WST  makes  any  but  the  most 
thoughtful  changes  to  a  planned  sceanario  risky 
and  argues  for  a  system  vrfiich  allows  thoughtful 
investment  of  instructor  efforts  as  a  course 
developer/mission  planner  before  the  student 
arrives. 

Thus  the  primary  goal  is  the  redrction  of 
instructor  task  loading  in  simulator-unique 
duties.  As  a  fallout  of  this  priority,  the  system 
further  must  assist  the  instructor  to  the  greatest 
extent  possible  in  the  gathering  of  a  broad  body 
of  data,  meaningful  in  the  evaluation  of  the 
trainees'  performance  not  only  in  terms  of  the 
quality  of  results,  but  also  useful  in  the 
diagnosis  of  problem  source  areas. 


FACTORS  INFLUENCING  THE  DESIGN  AND  DEVELOPMENT 

Early  in  the  analysis  phase  it  became  evident 
that  a  number  of  conflicting  factors  would  affect 
the  design,  development,  and  ultimate  performance 
of  the  Instructional  System. 

Since  it  was  determined  that  the  primary  em¬ 
phasis  would  be  placed  on  usability  of  the  system 
by  the  instructors,  control/operation  was  an  area 
of  concern.  It  was  once  considered  "sound"  human 
engineering  to  approach  design  of  controls  from 
the  "form  follows  function"  position  and  provide 
different  types  of  controls  for  dedicated  func¬ 
tions  (e.g.,  automobile  windshield  washer/wiper 
controls  should  differ  from  exterior  light  con¬ 
trols).  Each  should  be  optimized  for  its  indivi¬ 
dual  functions  but  sufficiently  different  so  that 
they  do  not  confuse  the  operator. 

However,  since  the  operation  of  each  control 
must  be  learned,  as  the  number  of  controls  and 
controllable  items  Increases,  consideration  of 
required  learning  time  quickly  diminishes  the 
desire  for  different  controls.  The  need  to 
simplify  operation  was  established  as  a  design 
goal.  As  a  result,  only  a  few  different  interface/ 
control  functions  were  specified  and  the  detailed 
system  design  was  forced  to  satisfy  requirements 
using  the  specified  functions.  Since  it  is  a  rela¬ 
tively  easy  software  task  to  monitor  inputs  for 
valid  form  and  sequence,  the  burden  of  Insuring 
that  inappropriate  or  accidental  use  did  not  re¬ 
sult  In  disaster  (or  frustrated  Instructors)  was 
placed  on  the  software.  Although  manual  control  of 
many  Items  was  required  (and  is  desirable  in  some 
instances)  the  need  to  "minimize  Instructor  task 
loading  by  automation  was  established  as  a  design 
goal.  The  Automatic  Mission  System  was  conceived 
as  the  mechanism  to  achieve  these  reductions  In 
task  loading  by  "programming"  events  and  sequences 
to  control  such  items  as  malfunction  Insertion  and 
deletion,  voice  message  activation,  procedure  and 
performance  monitoring,  external  environmental 
conditions  control,  and  CRT  display  updating. 

The  key  to  developing  a  useful  automatic  mis¬ 
sion  is  in  planning.  Since  instructors  would  be 


the  planners  and  developers,  the  system  design 
approach  required  data  generation  procedures  in  a 
form  usable  by  instructors  rather  than  pro¬ 
grammers. 

Because  of  the  competitive  nature  of  the 
program,  the  WST  had  to  be  operational  for  the 
start  of  customer  testing.  Therefore,  the  schedule 
became  the  primary  driver. 

Also  because  of  the  competition,  design  di¬ 
rection  and  approval,  normally  a  customer  func¬ 
tion,  was  replaced  by  a  “review  and  comment" 
position  by  the  Air  Force,  and  responsibility  for 
design  decisions  was  a  contractor  function. 

Another  factor  trfiich  influenced  the  B-52  WST 
development  was  the  contract- required  development 
of  a  KC-135  trainer.  The  Instructional  System  spe¬ 
cifications  for  the  KC-135  were  very  similar  to 
those  in  the  B-52  specification.  However,  the  de¬ 
tailed  simulation  and  training  systems  functions 
were  considerably  different.  This  implied  the 
highest  feasible  degree  of  commonality  among 
systems. 

Another  consideration  was  the  major  and  minor 
system  updates  which  might  be  required  to  keep  the 
B-52  viable  during  the  next  decade  or  two.  These 
would  require  associated  modification  of  the  WST, 
implying  a  high  degree  of  flexibility. 

Additionally,  the  system  complexity  alluded 
to  previously  in  the  summary  of  requirements  would 
be  a  prime  factor  affecting  the  development 
process.  Although  not  a  direct  concern  of  the 
subject  at  hand,  since  the  majority  of  work  was  in 
software  development,  this  deserves  at  least  a 
brief  discussion. 

Normal  software  development  practice  requires 
that  the  building  of  a  system  proceed  through  dis¬ 
tinct  phases: 

1.  Determine  system  requirements 

2.  Formulate  design  concept/approach 

3.  Develop  preliminary  design,  function 
allocation,  and  interface 

4.  Formulate  system  test  approach 

5.  Develop  detail  design/test  criteria 

6.  Code/test  components 

7.  Install/verify  system 

8.  Integrate/test  system  with  other  systems 

Although  there  Is  always  some  apparent  retrograde 
motion  between  phases  throughout  the  development 
process,  the  process  is  manageable  with  appro¬ 
priate  milestones  and  constraints  Identified. 

This  normally  smooth  process  was  complicated 
by  the  fact  that  to  meet  the  schedule  it  was  neces¬ 
sary  to  start  software  design  and  development  be¬ 
fore  the  instructional  system  analysis  had  com¬ 
pletely  defined  the  system  requirements.  A  system 
which  maximized  designed-in  flexibility  was 
necessary  to  accommodate  late-arriving  require¬ 
ments  as  well  as  any  future  modifications.  At  the 
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same  time,  as  much  commonality  among  the  B-52/ 
KC-135  stations  as  possible  seemed  a  necessity. 

The  general  approach  taken  to  developing  the 
software  drew  from  the  concepts  of  top-down 
design,  structural  programming,  and  programming 
teams.  However,  much  of  the  rigor  associated  with 
these  techniques  was  not  applied.  Software  design 
started,  by  necessity,  at  the  top  by  developing  a 
structure  which  could  accommodate  anticipated 
requirements  and  perform  anticipated  functions 
with  minimum  impact  on  computer  resources.  Gener¬ 
alized  systems  were  developed  by  a  team.  This  pro¬ 
vided  several  advantages.  Programmers  obtained 
general  overall  knowledge  of  several  systems  pre¬ 
paring  them  for  the  development  to  come.  As  more 
specific  requirements  were  supplied  their  imple¬ 
mentation  didn't  require  starting  from  scratch. 
Also,  as  priorities  changed  it  was  easier  to 
adapt. 

Without  the  rigor  of  many  detailed  design  re¬ 
views  and  accompanying  documentation  to  provide 
management  visibility,  it  became  a  frustrating 
environment  to  manage. 

INSTRUCTIONAL  SYSTEM  OVERVIEW 


In  reviewing  the  WST  system,  it  is  useful  to 
consider  the  basic  structure  of  the  Instructional 
System,  and  consider  some  of  the  most  Influential 
of  the  system's  features. 

The  WST  instructional  system  is  based  on  a 
remote  instructor  position  as  the  primary 
operating  location.  An  on-board  position  is 
provided,  but  is  viewed  only  as  a  secondary 
location.  The  remote  Instructor  console,  shown  in 
Figure  4,  provides  the  instructor  with  extensive 
control  and  monitor  capabilities.  The  console  con¬ 
sists  of  a  functional  keyboard,  which  is  used  to 
command  the  display  of  various  "pages"  on  the  two 
CRT's  of  the  display  system.  Switchlight  controls 
in  the  panel  provide  control  and  status  infor¬ 
mation  for  various  functions  such  as  the  motion 
system,  trainer  mode,  and  the  instructor  com¬ 
munication  system.  Monitors  allow  the  Instructor 
to  monitor  scenes  displayed  to  the  trainee  by  the 
visual  or  radar  systems. 

In  order  to  describe  in  a  useful  way  the 
characteristics  of  the  system,  four  particularly 
significant  views  are  presented,  each  looking 
primarily  at  a  key  system  feature. 


PANEL  FIS  PANEL  OIS 


1.  VISUAL  REPEATED 
EVS  MONITOR 
SPEAKER 

2.  GRAPHIC  CRT 

3.  GRAPHIC  CRT 

«.  AIR  REFUEL  CONTROLS 

VISUAL/EVS  CONTROLS 
MOTION  CONTROLS 
MASTER  OATA  RESET 
STATUS/CAUTION  ITS. 
LIGHTING  CONTROLS 
EMERGENCY  STOP 

S.  FUNCTION  KEYBOARD 

SIMULATOR  CONTROLS 
TRAINING  CONTROLS 
CRT/KYBD  ASSIGN 

6  JOYSTICK/SELECTION 

COMM  SYSTEMS  CONTROLS 


1 .  RADAR  REPEATER 
SPEAKER 

2 .  GRAPHIC  CRT 

3.  GRAPHIC  CRT 
«. 

EVS  CONTROLS 
MOTION  CONTROLS 
MASTER  OATA  RESET 
STATUS/CAUTION  LTS 
LIGHTING  CONTROLS 
EMERGENCY  STOP 

5.  FUNCTION  KEYBOARD 
SIMULATOR  CONTROLS 
TRAINING  CONTROLS 
CRT/KYBD  ASSIGN 

6.  JOYSTICK/SELECTION 
COMM  SYSTEM  CONTROLS 


Figure  4  B-52  REMOTE  INSTRUCTOR  CONSOLE 


The  most  fundamental  area  of  interest  is  the 
operation  of  the  display  system  and  keyboard. 
Since  the  system  reliability  is  of  pervasive  in¬ 
terest,  virtually  all  status  displays  and  control 
functions  are  accessed  via  the  CRT  system  in  order 
to  avoid  the  maintenance  problems  associated  with 
hardware  instrument  repeaters  and  controls.  The 
Flight  Instructor  Station  keyboard  is  shown  in 
Figure  5.  The  keyboard  system  employs  a  very  sim¬ 
ple  operating  scheme.  The  main  keyboard  is  used  to 
command  the  display  of  CRT  "pages"  by  a  single  key 
selection. 

Keys  are  grouped  into  various  classes  by 
location  and  a  color  code  scheme.  The  left  area 
contains  white  programming  keys,  used  for  control 
and  setup  features  such  as  initialization,  visual 
control,  and  various  atmospheric  conditions.  These 
are  prime  candidates  for  automation  to  reduce 
control  task  loading.  In  order  to  simplify  the 
page  fetch  operaion,  many  categories  use  an  index 
scheme  to  allow  quick  advance  to  the  desired  paae. 
The  keys  on  the  upper  right  are  color-coded  blue 
and  command  the  display  of  plot  function  pages. 
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These  provide  a  maplike  display  of  the  simulated 
aircraft's  position.  This  type  of  display  Is  an 
aspect  of  a  trainer  which  goes  beyond  real-world 
hardware,  since  flight  Instructors  rarely  have  ac¬ 
cess  to  such  an  easily  used  display  of  positional 
Information. 

The  remainder  of  the  main  keyboard, 
color-coded  in  grey  and  black,  houses  keys  which 
command  the  display  of  status  data.  The  black-key 
pages  contain  training  sortie  data  such  as 
automatic  mission,  automatic  performance  monitor, 
procedure  (checklist)  monitor,  or  malfunction 
system  status  displays.  These  are  trainer-unique 
features  which  are  classes  of  Information  not 
available  in  real-world  flight.  The  grey-key  pages 
allow  the  instructor  to  monitor  on-board  Instru¬ 
ments  and  switches.  The  use  of  CRT  displays  for 
this  purpose  provides  acurate,  easily  accessed 
data  with  high  reliability.  Comprehensive  coverage 
Insures  that  the  Instructor  can  monitor  cockpit 
activities  without  actually  being  present  in  the 
cockpit,  thus  avoiding  the  effect  of  an  Instruc¬ 
tor's  presence. 

Although  each  of  the  CRT  page  formats  shown 
in  Figure  6  is  identified  as  a  programming, 
status,  or  plot  style,  these  distinctions  are 
derived  more  from  customary  use  than  from  any 
system  contralnt.  Different  paging  (forward  and 
back)  advantages  Inherent  In  these  styles  provide 
for  their  main  differences.  Ths  r'\  ERT"  area  on 
each  provides  a  non- page  dependent  area  for 
Important  messages  alerting  the  Instructor  to 
system  activity,  such  as  motion  system  warnings, 
automatic  system  events,  c>  other  key  milestones. 
Rather  than  discuss  the  i^eclflc  content  of  the 
CRT  page  collection,  a  category  list  has  been 


included  In  Table  2  to  provide  some  Idea  of  the 
relative  size  and  diversity  of  the  library  with 
which  the  Instructor  Interacts. 


TABLE  2  DISPLAY  PAGE  CATEGORIES 


PROGRAMMING 

STATUS 

MISSIONS 

F  FLIGHT  INSTRUMENTS 

INITIAL  CONDITIONS 

F  AIRCRAFT  SYSTEMS 

AIRFIELDS 

F  ENGINE/FUEL 

F  VISUAL/E VS 

F  WARNING  LIGHTS 

PWT 

F  AUTO  PI  LOT/ E VS 

F  FUEL 

0  RADAR/EVS 

WEAPONS 

0  NAV/ASTROCOMPASS 

0  CELESTIAL 

0  BOMB/AGM 

0  AUTO  TO  AND  LND 

0  BOMB  SCORE 

0  FLIGHT  CONTROL 

0  AGM  SCORE 

PERFORMANCE  MONITOR 

PERFORMANCE  MONITOR 

EVALUATION  CRITERIA 

MISSION 

HARDCOPY 

WEAPON 

MALFUNCTIONS 

NAV/COMM 

PROCEDURES 

PROCEDURES 

MOTION  EXPOSURE 
PARAMETER  FREEZE 

MALFUNCTIONS 

POSITION  RECORD 

PLOT 

PLAYBACK 

F  MESSAGE  RECORD 

X-C  NAVIGATION 

0  RADAR  PREDICTION 

ILS/PAR 

0  SRAM  BY-PASS 

APPROACH/OEPARTURE 

0  BNS  JAMMING 

RENDEZVOUS 

WEAPON  DELIVERY 

F  -  FLIGHT  ONLY  0  -  OFFENSIVE  ONLY 


*  ALERT 

■ 

ALERT 

I 

MAIN  DATE  AREA 

I 

MAIN  DATA  AREA 

INDEX 

AND 

SUPPORT 

DATA 

1 

AREA 

(PAGE  X  OF  Y) 

■ 

(PAGE  X  OF  Y) 

EDIT  AREA 

1 

EDIT  AREA 

FULL  PAGE  PROGRAMMING  FORMAT  SPLIT  PAGE  PROGRAMMING  FORMAT 


Figure  6  CRT  PAGE  FORMATS 
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The  wide  range  of  CRT  pages  provides  for 
Interesting  diversity,  but  without  the  ability  to 
interact,  instructors  are  certainly  not  going  to 
have  the  control  they  need.  The  small  numeric 
keypad,  seen  In  Figure  5  at  the  extreme  right,  and 
the  two  rows  of  numbered  special  function  keys 
along  the  top  of  the  functional  keyboard  provide 
this  ability.  Each  CRT  page  item  which  may  be 
modified  Is  Identified  by  an  “Item  number." 

Where  variable  data  is  to  be  edited,  such  as 
in  the  specifying  of  visibility  (for  visual 
scene),  the  item  number  is  Identified  via  the 
numeric  keypad.  This  causes  the  item  of  interest 
to  be  displayed  in  the  Edit  area.  New  data  may 
then  be  keyed  into  the  system.  Here  is  where  vari¬ 
ous  checks  are  made  to  detect  errors  of  format, 
number  of  characters,  and  magnitude  of  the  input. 
In  the  event  that  an  error  is  detected,  the  input 
is  rejected,  and  an  error  message  appears  to  in¬ 
form  the  user  of  the  error.  It  is  this  safeguard¬ 
ing  of  input  at  the  keyboard,  along  with  the 
voiding  of  any  fatal  keystroke  sequences,  which 
provide  the  foundation  for  system  reliabilty. 

Just  as  many  data  items  require  the  Input  of 
numeric  values,  many  require  merely  the  selection 
between  two  states  or  from  an  index.  This  is  an 
area  where  the  UST  instructional  system  departs 
from  the  specification  in  the  dominance  of  the 
special  function  keys  rather  than  of  a  light  pen. 
The  use  of  light  pens  is  fairly  common,  and  is  de¬ 
scribed  in  the  WST  specification.  The  Flight  and 
Offensive  Instructor  Stations,  however,  exclusive¬ 
ly  rely  on  special  function  keys  to  allow  selec¬ 
tion  from  menus  or  to  switch  between  alternate 
states  of  boolean  items.  The  advantages  of  this 
alternative  include  increased  reliability  and  ease 
of  use,  reduced  instructor  hand-arm  motion,  and 
lower  life-cycle  cost. 

Virtually  all  system  transactions  are  con¬ 
ducted  via  these  two  media,  all  conforming  to  a 
common  set  of  format  schemes.  Table  3  enumerates  a 
display  and  control  system  summary,  with  operator 
options  and  CRT  system  capabilltes.  Although  the 
number  of  interactive  fields  is  large  and  the 
different  parameters  themselves  varied,  only  a 
small  number  of  distinct  data  interaction  formats 
are  permitted.  This  commonality  of  syntax  directly 


TABLE  3  INSTRUCTOR  DISPLAY/CONTROL  SYSTEM 


FUNCTION  KEYBOARD 

ALPHA-NUICR I C/GRAPH I C  CRT's 

0  PAGE  SELECTION 

0  FORMATS 

PAGE  FORWARD /PAGE  BACK 

FULL  PAGE  (12"H  X  16"W) 

RESET  PAGE 

SPLIT  PAGE  (12"H  X  1Z"W, 

DISPLAY  SWAP 

12"H  X  4"W) 

0  DATA  MODIFICATION 

0  PAGE  TYPES 

NUMERIC  PAD 

INDEX 

CLEAR/TAB/RUBOUT 

SINGLE  PAGE 

INSERT/DELETE/SAVE 

MULTIPLE  PAGE 

0  SPECIAL  FUNCTION  KEYS 

0  PAGE  CATEGORIES 

PAGE  SELECT  VIA  INDEX 

PROGRAMMING 

DATA  MODIFICATION 

STATUS 

SUB-SYSTEM  CONTROL 

PLOT 

0  DIRECT  DATA  ACCESS 

0  APPLICATIONS 

MALFUNCTIONS 

SET-UP  AND  CONTROL 

PROCEDURES 

EQUIPMENT  STATUS 

MESSAGES 

TRAINING  STATUS 

■ 

MISSION  PHASE  PROFILES 

supports  the  reliability,  ease  of  operation,  and 
speed  of  training  goals  defined  for  this  system. 
This  allows  him  (or  her)  to  actually  begin  using 
the  trainer  and  develop  "on  the  job."  This  mini¬ 
mizes  the  amount  of  system  operation  training  re¬ 
quired  so  that  the  bulk  of  an  instructor's 
training  can  be  devoted  to  the  effective  use  of  a 
synthetic  trainer  as  a  training  aid. 

Another  key  area  of  WST  performance  is  In  the 
direct  control  by  instructor/operators  of  the  mode 
or  degree  of  Integration  of  the  WST  stations. 
Instructor  control  of  the  trainer  must  be  simple, 
quick,  minimally  error-prone,  and  most  Important, 
extremely  reliable.  The  system  must  also  insure 
that  no  instructor's  lesson  is  interrupted  by 
another  station  operator.  The  system  developed 
reflects  these  requirements  in  Its  operation.  Mode 
changes  in  an  upward  direction  —  I.e.,  Indepen¬ 
dent  to  Integrated  (all  three  stations  together) 
—  require  the  consent  of  the  instructor  at  each 
involved  station.  No  effects  of  a  mode  change  are 
felt  at  a  station  until  its  Instructor  consents  by 
depression  of  a  mode  selection  swttchlight. 

Three  switchl ights,  one  for  each  mode,  are 
used  at  each  station.  The  software  which  operates 
behind  this  simple  exterior  is  of  considerable 
complexity.  Since  a  number  of  request  and  status 
flags  are  used  to  communicate  each  station's 
situation,  and  further,  since  a  given  station  may 
experience  momentary  losses  of  communication  with 
the  others  (or  complete  failure),  the  transfer  of 
status  data  is  continuous  rather  than  based  on 
change.  This  precludes  any  "out  of  sync"  situation 
due  to  a  missed  signal.  Further,  automatic  "down- 
mode"  is  designed  in.  If  a  system  failure  occurs 
at  one  station,  the  remaining  “healthy"  systems  do 
not  continue  to  look  for  data  from  the  failed 
station,  but  rather  step  down  to  the  next  lower 
mode. 

Another  key  software  characteristic  is  a  data 
priority  system.  Since  each  station  has  the  cap¬ 
ability  of  operating  as  a  stand-alone  device,  each 
computes  all  parameters  required  for  Its  simula¬ 
tion  environment.  Each  station  has  specialized 
areas  where  greater  detail  Is  required,  but  in 
general,  when  the  stations  are  combined  to  form 
the  Integrated  WST,  many  parameters  are  computed 
in  all  three  computer  complexes.  Consider 
latitude/longitude  and  weapon  load.  All  three 
stations  compute  position  based  upon  flight  vector 
inputs.  Both  the  Flight  Station  (for  weight  and 
balance)  and  the  Offensive  Station  (primarily  for 
weapon  delivery  gear)  maintain  a  library  of  weapon 
loads  and  track  real-time  weapon  status.  To  avoid 
confusion  and  anomalies  during  jolnups  and  for 
Initializations  in  Integrated  mode,  priority  is 
assigned  for  each  parameter  to  the  station  most 
concerned  or  involved  with  that  given  piece  of 
data.  In  our  examples.  Flight  controls  the 
latitude/longitude  pair,  while  Offensive  controls 
the  weapon  load.  A  system  Is  currently  envisioned 
which  allows  the  designation  of  parameter  pri¬ 
orities  in  real  time  by  the  Instructor.  This 
promises  to  Inject  even  more  flexibility  Into 
"mullt-mode"  mission  jolnups. 

In  the  course  of  the  discussion  we  have  seen 
that  a  considerable  portion  of  the  CRT/keyboard 
system  capability  is  utilized  for  control  of  the 
simulation  and  training  envlroment.  The  desire  for 
a  system  with  wide  ranging  flexibility,  very  slm- 
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pie  operation,  and  low  Instructor  task  loading 
provides  a  considerable  challenge. 

The  extensive  use  of  automation  in  eliminat¬ 
ing  instructor  interaction  and  reducing  task  load¬ 
ing  in  the  setup  and  control  of  the  trainer's  sim¬ 
ulated  environment  was  desired  by  the  Air  Force. 
This  is  an  area  where  the  flexibility  for  system 
configurations  and  training  session  events  is  of  a 
strategic  rather  than  tactical  nature.  By  using  a 
system  which  formalizes  required  data  specifica¬ 
tions,  but  allows  a  wide  variety  of  data,  one  can 
reduce  the  Instructor's  real-time  task,  provide 
great  system  flexibility,  but  also  allow  for  ex¬ 
tensive  ccmpatibil ity  testing  and  development  of 
the  situations  produced.  The  embodiment  of  this 
concept  is  the  use  of  a  mission  data  base  to 
specify  the  environmental,  task-related,  problem- 
related,  and  evaluative  characteristics  of  a 
training  session. 

By  setting  forth  the  characterises  of  a  ses¬ 
sion  in  advance,  the  complex  interelationships  in 
the  crew  task  structure  may  be  monitored  through 
the  thoughtful  creation  of  a  problem  scenario, 
complete  with  appropriate  procedural  and  per¬ 
formance-based  evaluative  monitoring. 

As  noted  earlier,  the  Intense  intertwining  of 
tasks  in  B-52  crew  activities  requires  that  train¬ 
ing  problems  and  monitoring  techniques  be  ex¬ 
tremely  well  planned  and  coordinated.  Automatic 
systems  to  monitor  crewmember  performance  in 
checklist  procedures,  weapon  delivery  accuracy, 
and  navigation  accuracy  against  recorded  parameter 
values  are  all  included  in  the  WST,  but  in  order 
to  control  these  features,  the  WST  Mission  System 
was  developed. 

The  Mission  System  may  be  viewed  as 
"automatic"  in  that  it  performs  a  host  of 
functions  with  virtually  no  help  from  the 
instructor.  During  a  training  session,  having 
selected  and  activated  one  of  the  available  mis¬ 
sions,  the  instructor  is  free  to  observe  as  the 
system  unfolds  the  problems  in  the  selected  lesson 
plan.  The  system  controls  the  insertion  and  dele¬ 
tion  of  malfunctions,  the  monitoring  of  checklist 
procedures  and  parameters,  and  the  transmission  of 
digital-voice  messages.  Other  options  allow  the 
control  of  simulated  environmental  systems  such  as 
visual  system  conditions,  wind,  atmospheric 
pressure,  turbulence,  and  temperature. 

The  system  operates  by  reducing  the  entire 
WST  mission  to  a  series  of  elements  called 
maneuvers  and  still  shorter  elements  known  as 
segments.  This  structure  may  be  seen  in  Figure  7. 
Maneuvers  are  "sub-missions"  which  include  a  com¬ 
plete  set  of  initialization  data.  This  allows  a 
mission  to  be  entered  at  any  maneuver  boundary 
rather  than  requiring  that  a  mission  always  start 
at  the  very  beginning.  The  basic  operating  unit  of 
the  mission  is  the  segment.  Segments  are  started 
and  ended  by  the  successful  fulfillment  of  con¬ 
ditional  statements.  Activities  within  a  segment, 
irfiether  it  be  a  malfunction  Insertion,  a  wind 
change,  or  a  parameter-set  activation,  are  trig¬ 
gered  by  the  reaching  of  some  parameter's  required 
value.  The  Mission  System  is  the  framework  of 
real-time  programs  which  operates  to  make  this 
happen.  The  real  structure  for  the  training  ses¬ 
sion  comes  from  the  data  upon  which  the  Mission 
System  operates.  The  parameters  to  monitor,  the 


malfunctions  to  Insert,  all  the  system  initializa¬ 
tion  data,  and,  most  importantly,  all  of  the 
trigger  parameters  and  threshold  values  are  the 
Mission  Data  Base. 

In  considering  the  Mission  Data  Base,  a  fun¬ 
damental  architectural  quality  of  the  WST  is 
illustrated.  Wherever  possible,  software  has  been 
developed  which  utilizes  a  data  base  in  carrying 
out  its  activities  rather  than  doing  so  through 
the  use  of  specific  data  implicitly  written  into 
the  simulation  programs.  Areas  which  utilize  this 
structure  are  numerous,  and  impact  the  total 
system  by  allowing  considerable  change  potential 
without  the  need  to  alter  simulation  code.  Imple¬ 
menting  this  scheme  required  that  real-time  pro¬ 
grams  fetch  and  operate  with  data  files  placed  on 
the  operating  disk-packs  by  off-line  data  genera¬ 
tion  processors.  The  system,  therefore,  cannot  be 
considered  as  the  real-time  control  programs, 
status  displays,  and  data  bases  alone,  but  must 
also  include  the  off-line  processors  which  build 
the  data  base,  and  the  planning  procedure.  This  is 
one  reason  why  the  "automatic"  Mission  System,  and 
hence  the  Instructional  System  as  a  whole,  is 
better  thought  of  as  "semi-automatic."  The  lure  of 
carefree  operation  of  the  real-time  lesson  carries 
with  it  a  subtle  but  inevitable  snare. 

The  Mission  lesson  plan  must  be  painstakingly 
and  precisely  prepared.  Events  are  planned  rrfiich 
develop  into  the  desired  scenario,  and  then  they 
must  be  predicated  on  appropriate  parameter 
triggers  so  that  the  mission  flows  in  spite  of 
student  errors.  Not  an  Impossible  chore,  but  one 
which  requires  complete  and  thoughtful  planning. 
Who  better  to  do  this  than  the  instructor  himself? 
Therefore,  the  Off-Line  Mission  Generation  proces¬ 
sors  accept  planning  data  vrfiich  has  been  transfer¬ 
red  directly  from  planning  forms  designed  to  be 
used  by  WST  instructors.  The  listing  produced  by 
the  processors  reprints  the  mission  data  base  with 
error  messages  for  Illogical  or  out-of-limit  in¬ 
puts  in  a  format  similar  to  that  used  on  the  form 
so  the  instructor  can  review  it  for  errors  and 
make  corrections  as  well  as  use  it  in  test- flying 
the  mission.  The  only  steps  in  mission  generation 
requiring  computer  support  are  the  running  of  the 
processors  and  copying  of  the  data  are  base  files 
to  the  real-time  operating  disk,  tasks  which  take 
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only  minutes.  No  software  support  personnel  are 
required.  Air  Force  instructors  using  the  system 
during  qualification  test  had  good  results  with 
extremely  short  training. 

An  added  dividend  from  the  use  of  this  struc¬ 
ture  Is  the  easy  adaptability  to  new  tactical 
scenarios,  new  aircraft  equipment,  and  develop¬ 
mental  error  correction.  Although  new  aircraft 
equipment  may  require  the  alteration  of  simulation 
programs  and  additions  to  Instruction  System  moni¬ 
toring  modules,  design  of  scenarios  exploiting  new 
capabilities  or  new  tactical  doctrine  is  easily 
accommodated.  This  important  feature  has  already 
proved  useful  during  VIST  development. 

Because  considerable  development  effort  has 
been  continued  throughout  the  testing  phase, 
various  bits  of  new  information  have  been  accumu¬ 
lated  which  caused  the  content  of  data  base  struc¬ 
tures  to  change.  The  use  of  this  structure  contin¬ 
ues  to  allow  a  high  degree  of  flexibility  in  the 
system,  and  provides  system  designers  with  the 
ability  to  defer  many  specific  scenario  decisions 
which  were  not  supported  by  solid  factual  or 
clearly  decisive  information. 

One  of  the  most  significant  shifts  in  WST 
design  philosophy  has  been  the  elevation  of  the 
Mission  System  as  the  prime  instructional  data 
source  for  the  trainer.  This  can  be  seen  from  two 
considerations. 

First,  the  complex  interaction  between  the 
task  activities  of  the  various  crewmembers  and  the 
high  level  of  simulation-unique  control  capability 
argue  for  a  carefully  planned  lesson  plan  to  fully 
exploit  the  system  even  for  the  most  basic  of 
part-task  training.  This  holds  true  for  just  about 
all  elements  of  the  operational  skills  inventory 
of  the  B-52  crewmember. 

Second,  the  Mission  Planning  task  has  proved 
to  be  easy  enough  to  permit  the  development  of 
mission  data  bases  for  almost  any  and  all  pur¬ 
poses.  The  mission  data  base  is  easily  adaptable 
to  a  data  base  consisting  of  segment  after  segment 
of  takeoffs,  refuelings,  bomb  runs,  etc.  The 
mission  data  base,  then,  is  more  accurately  viewed 
as  a  "training  sortie"  data  base  rather  than  being 
limited  to  a  “mission"  which  brings  to  mind  the 
typical  Strategic  Air  Command  sortie.  Thus  the  WST 
has  emerged  as  a  "mission"  trainer,  with  training 
sessions  based  upon  carefully  planned  lessons, 
automated  execution  of  these  lesson  plans,  and 
maximum  use  of  the  instructor/ operator  for  fine- 
tuning  of  the  trainer's  responses  and  evaluative 
observation. 

In  order  to  meet  the  challenges  of  the  B-52 ' s 
complex  instructional  requirements,  the  automation 
of  training  lessons  was  required.  With  this  comes 


the  need  for  careful  and  insightful  planning. 
Although  we  offer  a  highly  structured  and 
promising  training  tool,  the  value  lies  in  the 
planning.  The  entire  thrust  of  automation  is  not 
only  structure  and  order,  but  also  the  freeing  of 
the  Instructor.  Here  we  can  see  the  other  chal¬ 
lenge.  The  Instructor  can  no  longer  satisfy  his 
responsibilities  by  operating  the  trainer,  since 
that's  being  done  for  him.  He  is  now  challenged  to 
observe,  diagnose,  and  advise  --  in  short,  to 
instruct.  This  is  a  second  reason  to  discard  the 
notion  of  the  "automatic"  instructional  system.  At 
best  one  can  strive  for  a  semi-automatic  system, 
and  the  best  system  is  only  a  tool  in  the  hands  of 
an  Instructor.  It  requires  two  men  in  the  "man/ 
machine  interface,"  one  at  plan-time,  and  one  at 
run-time.  Both  play  such  an  intimate  role  in  the 
success  of  a  training  exercise  that  it  is  not 
unfair  to  regard  them  as  parts  of  the  overall 
system. 

Although  the  competition  is  over,  there  is 
continued  development  in  the  WST,  making  improve¬ 
ments  in  the  design  for  follow-on  production 
units.  Two  key  design  enhancements  which  are  anti¬ 
cipated  will  be  briefly  presented  to  conclude  this 
report. 

Observations  and  comments  during  Air  Force 
qualification  testing,  both  from  test  crews  and 
from  previously  untrained  Link  personnel,  seem  to 
indicate  that  in  spite  of  the  volume  of  data 
available  to  the  instructor*,  the  CRT  Display/ 
Function  Keyboard  system  operation  is  relatively 
easy  to  learn.  However,  to  properly  utilize  the 
available  data  in  a  training  situation  will  un¬ 
doubtedly  require  a  significant  learning  period. 
Two  basic  improvements  to  this  system  have  been 
proposed:  1)  a  rearrangement  of  page  access  keys 
to  provide  more  obvious  and  quicker  access  to  the 
most  often  used  pages,  and  2)  a  specially  format¬ 
ted  instructor  handbook  to  aid  in  using  the  system 
capabilities  more  effectively.  The  automatic  pro¬ 
cedure  monitoring  system,  although  compliant  with 
the  specification  requirements,  did  not  seem  to 
provide  sufficient  information  for  an  instructor 
at  the  remote  console  to  determine  the  student's 
actions  in  performing  many  of  the  procedures.  For 
example,  since  the  procedure  monitor  only  observed 
the  programmed  checklist  steps  and  logged  them 
when  detected  as  complete,  the  student  could  in¬ 
advertently  operate  other  controls  which  affected 
his  performance  on  the  procedure  without  the  in¬ 
structor  being  aware  of  the  action.  Also,  when 
considering  the  large  number  of  published  pro¬ 
cedures,  about  143  on  the  Flight  Station  alone, 
and  the  probable  future  change  activity,  keeping 
the  procedure  system  current  with  the  aircraft 
would  become  a  major  effort. 

An  alternate  system  which  meets  some  but  not 
all  requirements,  but  which  promises  to  alleviate 


•The  CRT  Display/Function  Keyboard  system,  which  is  the  primary  man-machine  Interface,  contained  the 
following  items  and  quantities  for  the  FIS  and  OIS: 


ITEM 

FIS 

OIS 

CRT  PAGES 

166 

139 

INTERACTIVE  ITEMS  (LESS  MALFS) 

914 

1055 

SYSTEM  MALFUNCTIONS 

1424 

629 

STATUS  ITEMS 

929 

1592 

TOTAL  #  VARIABLES 

3267 

3276 
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most  of  the  problems  whith  the  current  system.  Is 
being  investigated. 


Although  configuration  control  of  software 
during  the  prototype  development  was  generally 
better  than  on  any  other  recent  simulator,  if  the 
full  complement  of  17  8-52  WST's  is  produced,  the 
need  for  a  much  more  automated  process  is  evident, 
and  such  a  process  is  being  pursued. 


The  design  and  development  of  a  Weapon 
Systems  Trainer  (WST)  for  the  B-52  in  the  envi¬ 
ronment  of  a  competitive  procurement  was  both 
challenging  and  frustrating.  Since  the  primary 
objective  of  the  WST  is  to  aid  SAC  Instructors  in 
the  continued  training  of  combat  crewmembers,  the 
success  of  the  trainer  and  the  fulfillment  of  the 
major  goals  will  be  best  judged  after  a  period  of 
actual  training  exposure.  We  are  anxiously  await¬ 
ing  the  opportunity  to  further  develop  this  system. 
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ABSTRACT 

This  paper  analyzes  the  methodology  used  for  scenario  development  as  learned  from  the  Fire- 
finder  Radar  Trainer.  First,  an  overall  concept  of  the  operational  and  maintenance  trainer 
Is  given  with  special  emphasis  on  the  scenario  training  package.  The  training  effectiveness 
of  the  system  Is  demonstrated  by  reviewing  the  transfer  of  knowledge  tests  conducted  wltb 
the  actual  system.  The  management  of  the  scenario  data  base  Is  described,  stressing  the  use 
of  the  Chief  Programmer  Concept,  the  reuseablllty  of  the  data  elements,  and  the  adherance  to 
an  effective  configuration  control  plan.  The  characterization  of  three  levels  of  training 
complexity  as  designed  In  the  scenarios  Is  also  given.  Next,  several  training  schemes  are 
suggested  that  make  the  maintenance  trainer  a  cost-effective  device.  Emphasized  are  special 
CAI/CMI  techniques  used  to  transform  the  simulator  Into  an  effective  trainer. 


INTRODUCTION 


The  purpose  of  this  paper  is  to  analyze  the 
methodolgy  used  to  develop  scenarios  for  the 
Firefinder  Radar  Trainer.  The  trainer  was 
developed  under  a  Naval  Training  Equipment  Center 
contract,  managed  by  PM-TRAOE,  for  the  Field 
Artillery  School  at  Ft.  Sill,  Oklahoma.  The 
scenario  is  that  group  of  data  necessary  to  pro¬ 
cess  a  student's  response,  cause  a  student  station 
stimuli  to  occur,  and  provide  error  and  message 
data  for  evaluation  at  the  instructor's  station. 
The  scenario  allows  the  student  to  learn  in  a 
truly  advanced  training  environment. 

Firefinder  Radar  Description 

The  Hughes  Aircraft  Company  AN/TPQ-36  Fire¬ 
finder  Radar  is  a  portable  surveillance  system 
used  for  detecting  mortar  projectiles.  This  radar 
provides  first  round  detection  and  pin  point  cal¬ 
culation  of  battery  and  impact  coordinates.  This 
information  is  displayed  on  a  tactical  map  so  that 
a  radar  operator  might  review  the  data  before 
transmitting  the  information  to  the  command  post 
via  the  TACFIRE  link.  Figure  1  shows  the  radar  as 
emplaced.  Figure  2  details  the  operational  panel 
within  the  shelter  unit. 

Task  and  Skills  Analysis 

The  tasks  and  skills  analysis  (TASA)  was 
performed  and  determined  the  following  training 
requirements  for  the  radar  operator  and  the 
organizational  maintenance  technician: 

Simulated  AN/TPQ-36  Operational  Exercises 

a.  Power-on  sequence 

b.  Computer  program  loading  operations 

c.  Initialization  operations 

d.  Hostile  fire  operations 

e.  Friendly  fire  operations 


f. 

ECM  countermeasures 

9- 

TACFIRE  operations 

h. 

System  shutdown  sequence 

i . 

Clearing  system  or  on-line  fault 
indications 

Simulated  AN/TPQ-36  Organizational  Level 

Maintenance  Exercises 

a. 

Loading  individual  off-1 ine  diagnostic 
tests 

b. 

Troubleshooting  central  processor 
unit/input  output  controller  and  memory 
malfunctions 

c. 

Troubleshooting  xmtr  low-voltage  assembly 
mal functions 

d. 

Troubleshooting  SYSTEM  +28  voltage 
di stribution 

e. 

Troubleshooting  ac  prime  power 
distribution 

f. 

Adjusting  shelter,  receiver/exciter,  beam 
steering  unit,  and  xmtr  low-voltage  power 
suppl ies 

9- 

Isolating  a  fault  to  a  defective  low- 
voltage  power  supply 

h. 

Aligning  the  signal  processor  analog-to- 
digital  (A/D)  converter 

i. 

Performing  the  WLU  map  position  (northing) 
al ignments 

Trainer  Design  Approach 

Based  on  the  TASA  requirements,  a  trainer  was 
developed  to  realistically  simulate  the  man/ 
machine  interface  found  by  the  operator  and  main- 

tenance  personnel  in  the  actual  radar  systems. 
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figure  J. 


Firefinder  Trainer 


Figure  4.  Operational  Panel  Simulator 


The  trainer  design  provides  the  capability  for 
control  of  six  student  stations  by  a  single 
instructor.  Any  combination  of  operator  and  main¬ 
tenance  technician  students  can  be  trained  with 
continuous  computer-managed  instructor  monitoring. 
See  figures  3  and  4  for  a  representation  of  the 
trainer  system. 

The  system  provides  the  capability  to  simu¬ 
late  the  Firefinder  shelter  and  trailer  indicators, 
switches,  meters,  data  readouts,  and  other  physi¬ 
cal  factors  necessary  to  enable  the  desired  train¬ 
ing  to  occur.  The  student  learns  the  various 
skills  using  a  self-paced  training  exercise 
selected  by  the  instructor  to  satisfy  the  current 
course  curriculum.  Once  the  exercise  is  initiated 
by  the  instructor,  the  computor  will  respond  to 
each  student  action  by  simulating  each  Firefinder 
functional  area. 

As  the  students  progress  through  the  exer¬ 
cises,  the  computor  provides  the  instructor  sta¬ 
tion  with  student  station  "alert11  data,  and 
detailed  “performance  evaluation"  data.  The 
monitoring  capabilities  include  detail  display  or 
printout  of  student  actions,  historic  student 
data,  and  continuous  display  and  recording  of  the 
student  progress  data.  These  features  help  to  min¬ 
imize  the  instructor  load. 

Role  of  Instructor 

The  instructor  has  the  option  of  freezing  the 
exercise,  communicating  any  learning  difficulties 
with  the  student  using  provided  audio  channels, 
and  then  continuing  the  exercise  from  the  point  of 
interruption  or  restarting  from  any  point  in  the 
exercise.  Any  errors  that  the  student  makes  are 
displayed  at  the  instructor's  station  along  with 
continuous  updating  of  the  student's  percent 
accuracy,  percent  complete,  and  percent  of  the 
time  standards  met  within  the  scenario- 
established  criterion. 

Software  Approach 

The  software  for  the  Firefinder  trainer 
incorporates  the  data-base-driven  approach.  In 
this  approach,  the  OPERATIONAL  COMPUTER  PROGRAM 
(OCP)  consists  of  generalized  processing  func¬ 
tions,  coordinated  and  directed  by  the  data  base. 
The  response  to  any  input  can  be  altered  simply  by 
changing  the  data  base.  This  data-base-driven 
design  provides  greater  flexibility  in  defining 
training  situations,  allows  diagnostics  to  be  done 
on-line,  and  greatly  reduces  costs  and  implementa¬ 
tion  time  for  modification.  Figure  5  illustrates 
the  data-base  concept. 

Many  changes  may  occur  over  the  life  cycle  of 
the  trainer  as  a  result  of  Firefinder  production 
changes  or  refinement  of  training  objectives. 

Thus,  update  costs  are  a  very  significant  design 
consideration.  The  data-base-driven  design  will 
accommodate  most  of  the  changes  by  data  base 
modification  which,  historically,  is  much  less 
expensive  than  software  modification. 

The  trainer  software,  which  makes  up  all  the 
modules  in  the  OCP,  performs  all  of  the  functions 
in  the  trainer  system  that  are  not  specific  to  a 
particular  training  objective.  Examples  of  the 
functional  capability  of  the  OCP  include  printing 
evaluation  reports  and  processing  the  scenario 


Figure  5.  Data-Base-Driven  Software  Design 

data.  Figure  6  is  a  top-level  heirarchy  chart 
describing  all  of  the  functional  areas  within  the 
OCP. 

Scenarios 

A  scenario  is  a  group  of  event  forms  vrfiich 
direct  the  fortran-hased  software  package  (OCP)  to 
drive  the  various  hardware  devices.  The  scenario 
is  the  data  that  the  OCP  uses  to  process  student 
responses,  and  is  training  objective  specific. 

The  scenario  data  base  is  that  accumulation  of  all 
the  training  exercises  which  interacts  with  the 
student  through  the  OCP.  Each  individual  scenario 
will  present  a  stimulus  to  the  student  and  provide 
the  appropriate  response  to  his  action  based  on 
the  desired  training  objectives  of  the  task  to  be 
taught.  The  scenario  data  base  is  structured  in 
groups  of  families  which  represent  the  operational 
modes  (initialization,  hostile,  friendly,  ECM, 
zones  and  combined)  and  maintenance  modes  (equip¬ 
ment  status  tests,  circuit  card  alignment,  fault 
isolation  tests,  and  power  status). 

FEATURES  OF  THE  DATA  BASE  PACKAGE 

The  Firefinder  trainer  scenario  package  was 
developed  by  borrowing  software  development  tech¬ 
niques  already  in  use  and  incorporating  novel 
concepts  to  improve  the  efficiency  of  the  author¬ 
ship.  This  section  will  highlight  the  CHIEF 
PROGRAMMER  CONCEPT,  the  scenario  family  breakdown, 
the  three  levels  of  training  complexity,  the 
reuseability  of  data  elements,  and  the  configura¬ 
tion  control  techniques.  These  principles  serve 
to  increase  the  efficiency  of  the  scenario 
creation,  and  aid  in  giving  the  program  management 
office  clearer  visibility  in  the  data  base 
development  phase. 

Chief  Programmer  Concept 

The  Chief  Programmer  Concept  was  first  devel¬ 
oped  at  International  Business  Machines  Corpora¬ 
tion.  The  data  base  generation  effort  has 
borrowed  heavily  from  this  scheme  of  group  organi¬ 
zation.  The  scenario  creation  responsibilities 
revolve  around  the  subject  matter  expert  (SME), 
who  is  typically  the  most  experienced  scenario 
creator  in  the  staff  (In  the  software  analogy  , 
this  would  be  the  chief  programmer). 


Figure  6.  OCP  Top  Level  Hierarchy  Chart 


The  SHE's  job  is  to  compile  the  research 
accomplished  by  others  on  the  operational 
characteristics  of  the  prime  system,  and  develop  a 
pivot  scenario  vrfilch  serves  as  the  skeleton  for 
all  other  scenarios  in  the  data  base  package. 

This  master  pivot  design  incorporates  all 
operational  emulation  characteristics  and  excludes 
specific  grading  criteria  and  student  timing 
requirements. 

Other  scenario  creators  in  the  group  observe 
and  aid  the  SME  In  this  pivot  design.  After  this 
Interaction,  they  will  have  the  knowledge  to  trans¬ 
form  the  top  level  pivot  design  into  the  individ¬ 
ual  scenario  versions  which  meet  the  specific 
training  objectives  required  of  the  system. 
Apprentice  scenario  creators  work  with  and  learn 
valuable  creation  skills  from  the  SHE  and  work 
directly  with  the  other  members  of  the  group  on 
the  total  data  base  design,  Instead  of  completing 
a  more  minor  design  task  on  their  own. 

Other  members  of  the  group  include  a 
librarian/technician,  test  coordinator,  and 
business  manager.  The  librarian  enters  all 
scenario  data  into  the  computer  and  performs  other 
clerical  and  configurational  assignments  for  the 
group.  The  test  coordinator's  job  is  to  interface 
between  the  scenario  creation  group  and  the  system 
test  personnel.  By  observing  the  design  method¬ 
ology  of  the  data  base  development,  he  can  com¬ 
municate  testing  criteria  to  the  test  group  so 
that  they  may  best  validate  specification 
adherence  and  exercise  repeatabil Ity.  The 
business  manager  has  significant  technical 


background  and  managerial  experience.  His  job  is 
to  act  as  group  manager,  but  not  as  the  technical 
design  director.  The  business  manager  typically 
schedules  time  and  activities  of  several  other 
groups,  usually  the  systems  test  group  and 
courseware  development  group.  He  also  presents 
progress  reports  at  the  numerous  program  and 
division  management  meetings  so  that  the  SME  can 
dedicate  most  of  his  time  towards  design  efforts. 

The  Chief  Programmer  Concept  promotes  optimum 
efficiency  in  scenario  authorship  due  to  maximized 
reuseability  of  the  data  elements  and  minimum  risk 
of  duplication  of  effort  among  individual  scenario 
creators.  This  results  in  substantial  savings  in 
core  requirements  and  development  time,  thus  any 
exercise  update  effort  is  greatly  simplified  due 
to  the  scenario  design  concepts  being  universally 
understood  throughout  the  group. 

Exercise  Families 

Another  method  used  in  data  base  generation 
is  the  family  design  approach.  Using  this  scheme, 
all  training  objectives  forwarded  by  the  TASA  are 
separated  into  functional  groups  called  families. 
Each  scenario  within  a  family  Incorporates  train¬ 
ing  objectives  which  are  logically  related.  The 
SME  then  designs  a  pivot  scenario  for  each  family 
while  the  apprentice  authors  alter  the  pivot  to 
suit  the  training  objectives  of  each  task  to  be 
demonstrated.  The  pivot  is  altered  by  adding 
specific  data  entry  grading,  target  profiles,  and 
customized  Computer  Assisted  Instruction  (CA1). 

In  this  way,  the  standard  operating  procedure  is 
universal  within  the  scenarios  of  a  family  and 
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grading  criteria  can  be  effectively  made  more 
stringent  from  one  scenario  to  the  next. 

The  pivot  for  each  family  serves  as  a  "free- 
play“  scenario  which  does  not  exhibit  lock-step 
training  objectives.  These  scenarios  allow  the 
student  to  energize  any  switch,  enter  any  function 
code,  run  any  program,  or  select  any  feature  nor¬ 
mally  found  on  the  prime  system.  No  grading  of 
data  entry  is  incorporated;  however,  any  normal ’y 
illegal  action  is  flagged  as  such. 

Fach  scenario  within  a  family  becomes  pro¬ 
gress. ve’y  more  difficult.  At  the  same  time  each 
scenario  is  composed  of  5  different  options,  each 
option  requiring  the  student  to  enter  slightly 
different  data.  However,  each  option  incorporates 
the  same  level  of  difficulty  as  other  options  in 
the  scenario.  This  provides  several  options  to 
the  student  for  reinforcement  training  at  the  same 
level  of  difficulty.  Figure  7  clarifies  the  multi¬ 
plicity  of  options  that  are  available. 
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Figure  7.  Multiplicity  of  Exercise  Options 


3  Levels  of  Complexity 


Each  family  incorporates  3  levels  of  training 
complexity  in  order  to  expose  the  student  to  pro¬ 
gressively  more  difficult  tasks.  The  grading  cate¬ 
gories  which  can  exist  in  all  three  levels  should 
be  described  first.  The  scenario  can  designate  a 
student  action  as  a  not-allowed  response,  which  is 
not  down-graded,  as  it  is  normally  a  part  of  the 
actual  system.  This  category  was  created  to  keep 
the  student  from  taking  alternate  paths  that  are 
not  included  in  the  training  objectives.  Once 
such  action  is  taken,  the  student  is  prompted  with 
the  letters  "NAR,"  which  forbids  proceeding  in  the 
exercise  until  cleared  from  the  display.  Two 
types  of  error  classifications  exist  that  are 
down-graded.  A  non-critical  error  is  that  cate¬ 
gory  designated  by  the  scenario  to  reflect  data 
which  was  incorrectly  entered  or  an  undesired 
action  which  was  taken  by  the  student.  A  critical 
error  is  that  category  reserved  for  switch  actions 
which  are  more  severe  in  nature. 


The  primary  level  presents  few  complex  tasks 
and  is  characteri zed  by  a  very  narrow  path  of 
acceptable  switch  actions,  with  undesired  paths 
being  not-allowed.  For  example,  the  single  target 
profile  is  characteristic  of  a  primary  level 
hostile  scenario.  Lenient  time  and  grading 


standards  are  incorporated  with  automatic  back 
sequencing  (for  example,  reprompting)  on  incorrect 
student  actions.  Expected  action  messages  are 
sent  to  the  instructor  for  each  step,  while 
instructional  messages  referring  to  the  appro¬ 
priate  technical  manual  page  are  sent  to  the 
student  if  an  error  is  made.  Critical  errors  will 
evoke  buzzer  alerts  and  caution  messages  for  these 
potentially  dangerous  or  harmful  actions. 


Intermediate  scenarios  are  characterized  by 
more  complex  learning  environments,  such  as 
multiple  target  profiles  in  single  mode  form 
(hostile  or  friendly  mode).  More  paths  that  were 
designated  not-allowed  in  the  beginning  scenarios 
are  now  possible,  and  the  student  has  the  flexibi¬ 
lity  of  learning  new  function  codes  that  are  nor¬ 
mally  a  part  of  the  actual  system  and  is  not 
down-graded  in  this  exploratory  training.  CAI 
messages  are  utilized  for  incorrect  responses  but 
no  automatic  back  sequencing  is  employed.  CAI 
messages  are  also  used  in  association  training. 

As  an  example,  the  student  is  told  at  the  begin¬ 
ning  of  the  exercise  to  switch  frequencies  if 
jamming  is  encountered.  Later  in  the  exercise, 
when  jamning  occurs,  he  is  graded  on  his  ability 
to  remember  those  instructions.  Intermediate 
scenarios  also  incorporate  tighter  time  standards 
than  those  used  at  the  primary  level. 


Advanced  level  scenarios  are  characterized  by 
a  task  complexity  level  equal  to  that  trfiich  might 
be  encountered  in  the  battlefield.  The  student  is 
presented  with  a  combination  of  situations  and 
tasks  that  were  previously  presented  in  singular 
form,  that  is,  he  is  now  experiencing  a  multi¬ 
stimulus-response  arena.  Typical  advanced 
scenarios  present  multiple  targets  with  jamming, 
priority  tracks,  censor  zones,  and  interlocked 
hostile  and  friendly  processing.  Students  are 
forced  to  make  decisions  based  on  environmental 
factors.  As  an  example,  he  should  know  to  change 
frequencies  when  jamming  occurs,  without  instruc¬ 
tional  messages.  All  paths  are  allowed  and  total 
system  emulation  is  incorporated.  Time  standards 
meet  the  ultimate  training  objectives  of  the 
course.  No  intervention  by  the  instructor  is 
available  via  manual  back  sequencing.  The 
advanced  scenarios  are  typically  run  as  profi¬ 
ciency  exercises  because  the  instructor  will  be 
paying  close  attention  to  the  percent  complete 
field  (Were  all  tasks  performed?),  the  percent 
accuracy  field  (Were  any  errors  made?),  and  the 
percent  time  field  (Was  the  student  fast  enough?). 
Based  on  this  performance  data,  the  student  will 
pass  this  family,  be  retested  on  another  profi¬ 
ciency  exercise,  or  be  rerouted  to  an  intermediate 
exercise. 


Emulation 

As  an  aid  to  scenario  generation,  the  OOP 
Incorporates  an  optional  emulation  of  all  switches 
found  on  the  trainer.  If  the  option  is  exercised, 
the  software  will  automatical ly  give  a  response  to 
the  student,  which  is  the  exact  simulation  of  the 
response  of  the  actual  system.  As  an  example, 
when  a  circuit  breaker  is  turned  off,  the  OCP  will 
automatically  extinguish  all  lights  on  the  trainer 
which  would  normally  be  powered  by  that  circuit 
breaker.  This  emulation  option  saves  the  scenario 
creator  from  having  to  define  the  normal  response 
to  a  switch  action.  If  the  scenario  creator 
desires  a  switch  response  different  from  that 
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exhibited  in  the  real  system,  the  emulation  option 
is  not  used,  and  the  desired  response  is  specified 
using  the  appropriate  equipment  forms.  Examples 
of  the  emulation  not  being  used  would  include 
faulting  of  hardware  and  customized  CAI. 

More  complex  operational  emulation  of  the  charac¬ 
teristics  of  the  prime  system  must  be  designed  by 
the  scenario  creator.  In  the  trainer,  cues  pre¬ 
sented  to  the  student  and  his  responses  are  iden¬ 
tical  to  the  actual  radar  system  and  are  repre¬ 
sented  uniformly  to  each  trainee.  Data  for  these 
scenarios  are  derived  from  the  actual  operation  of 
the  AN/TPQ-36  radar  system  which  is  currently 
being  developed  by  Hughes  Ground  Systems  Group. 

Both  the  operational  and  maintenance 
scenarios  simulate  the  operation  of  all  the  func¬ 
tion  codes  (which  are  used  to  activate  the  various 
features  of  the  system).  The  alphanumeric  and 
graphic  simulation  includes  B-Scope  zones,  header 
messages,  function  code  prompts,  and  line  priority 
cues.  The  trainer  also  Incorporates  timing  pre¬ 
cision  comparable  to  the  prime  system.  This  can 
be  observed  in  the  length  of  the  computor  runs, 
antenna  slews,  buzzer  characteristics,  and  target 
tracking  profiles.  The  prime  system  incorporates 
a  legality  check  of  the  operator  entered  data  to 
determine  if  the  information  is  in  acceptable 
form.  Illegal  entries  are  thus  flagged  to  the 
operator  via  the  data  displays.  The  trainer 
emulates  these  legality  checks,  and  down-grades 
illegal  entries  as  errors  and  then  notifies  the 
instructor.  The  trainer  maintains  integrity  of 
all  switch  and  lamp  emulation  found  on  the 
Firefinder  Radar.  As  an  example,  when  a  switch  is 
pressed,  the  scenario  gives  a  reaction  to  the 
student  in  the  form  of  lit  indicators,  data 
prompts,  and  buzzer  cues.  Included  in  this  switch 
emulation  is  incorporated  a  legality  check  similar 
to  that  previously  mentioned  for  data  entries. 

The  scenarios  were  designed  so  the  student  is  able 
to  use  the  prime  system  technical  manual  through¬ 
out  the  exercises  in  order  to  become  familiar  with 
their  format  and  content. 


difficulties,  he  can  intervene  in  scenario  pro¬ 
cessing  by  freezing,  back-sequencing,  zeroing 
(restart  at  beginning),  or  aborting  the  exercise. 

The  summary  area  of  the  instructor's  display 
is  also  enhanced  by  an  inactivity  alert  for  each 
student  station.  If  the  student  fails  to  make  a 
switch  action  within  a  specified  period  of  time, 
the  instructor  is  alerted  that  the  student  has 
become  indecisive  and  is  probably  lost.  In  the 
same  way  if  the  student  makes  a  number  of  consecu¬ 
tive  errors  which  exceed  a  scenario  specified 
threshold,  then  the  processing  will  send  a 
"Consecutive  Error  Alert"  to  the  instructor 
station.  Exercise  grading  allows  some  errors  to 
be  classified  as  critical.  These  are  used  to 
down-grade  students  who  make  switch  actions  which 
might  harm  the  equipment  or  cause  a  safety  hazard, 
if  done  on  the  prime  system.  These  critical 
errors  also  appear  in  the  instructor’s  notice 
field. 

The  detailed  display  area  at  the  instructor's 
station  typically  lists  all  switch  actions  and  data 
entries  performed  at  a  particular  student  station. 
This  display  is  enhanced  by  scenario  control  of 
two  of  its  fields.  Stimuli  messages  show  the 
Instructor  a  summary  of  the  training  conditions 
being  presented  to  the  student.  If  a  student  has 
made  an  error,  the  expected  action  field  Informs 
the  instructor  *rt>at  correct  response  should  have 
been  taken.  Figure  8  illustrates  the  instructor's 
station  console  and  the  scenario  enhanced  fields 
of  information.  In  order  to  assess  a  student's 
speed  in  performing  a  training  task,  scenario 
controlled  clocks  are  utilized  to  determine  the 
time  interval  between  switch  actions.  If  the 
student's  time  to  perform  a  task  is  longer  than 
that  required  by  the  training  objective,  his  time 
criteria  score  is  reduced,  and  a  notice  is  gener¬ 
ated  in  the  instructor's  detailed  display.  See 
figure  9  for  a  CA1/CMI  feature  summary. 


Training  Effectivess  Tests 


While  accurate  emulation  of  the  operational 
characteristics  of  the  prime  system  is  important 
in  the  design  of  a  simulator,  enhancement  of  the 
simulation  is  needed  for  training  purposes.  An 
effective  trainer  is  much  more  than  just  a  good 
simulator.  The  approach  used  makes  extensive  use 
of  computer  assisted  instruction  (CAI)  and  com¬ 
puter  managed  instruction  (CMI).  The  CAI  Inherent 
in  the  scenarios  gives  the  student  simulated 
supervisor  messages  and  task  directions  at  appro¬ 
priate  points  in  the  exercise.  As  an  example, 
after  the  student  has  correctly  initialized  the 
radar  system,  he  is  alerted  with  a  buzzer  cue  and 
prompted  with  the  message  "Supervisor  commands  you 
to  begin  radiating".  It  is  Important  to  under¬ 
stand  that  an  operator  in  the  battlefield  cannot 
engage  this  feature  without  direction  from  his 
supervisor.  By  the  scenario  prompting  the 
student,  the  instructor  Is  freed  from  the  burden 
of  cueing  the  student  throughout  the  training 
lesson.  CAI  techniques  are  also  used  In  branching 
the  scenario  processing  backwards  to  reprompt  a 
student  with  the  original  stimuli  if  the  student's 
reaction  was  an  error.  Scenario  CMI  is  evident  in 
the  many  messages  sent  to  the  Instructor's  console 
when  a  student  has  made  an  error.  Once  the 
instructor  has  been  made  aw are  of  the  student's 


The  approach  taken  to  building  an  effective 
trainer  was  validated  by  a  series  of  government 
conducted  training  effectiveness  tests  (TET).  The 
TET  was  performed  on  the  operational  scenarios 
with  20  students  over  10  days  under  a  well- 
structured  test  environment.  The  tests  covered 
exercises  from  initialization  through  the  various 
hostile  modes.  The  first  important  result  of  the 
tests  is  that  the  correct  number  of  different 
versions  were  chosen  for  each  exercise.  During 
the  design  phase.  It  was  decided  that  for  remedial 
and  reinforcement  purposes  5  versions  of  the  same 
exercise  were  necessary.  Results  of  the  TET 
showed  that  a  learning  plateau  trend  was  exhibited 
at  the  4th  of  5  possible  versions. 

A  high  transfer  of  knowledge  was  displayed  by 
the  students  who  learned  on  the  Firefinder  Trainer 
Device  versus  the  control  students  in  the  test. 

As  an  example,  the  test  group  performed  the  Ini¬ 
tialization  task  on  the  prime  system  an  average  of 
32%  faster  than  the  control  group  with  no  appreci¬ 
able  difference  in  error  rates.  The  more  inter¬ 
active  exercises  strengthened  this  learning  trend 
in  an  even  greater  way.  The  test  group  performed 
the  hostile  mode  tasks  an  average  of  47%  faster 
than  the  control  group  and  at  the  same  error  rate. 

A  single  Firefinder  Trainer  device  has  the 
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Figure  8.  Instructor’s  Station  Console  Display 


Figure  9.  Dual  Purpose  Computer  Instruction 


capability  of  training  6  students  with  2  instruc¬ 
tors.  Because  of  a  logically  arranged  instructor 
station  control  format,  one  instructor  can  be 
allowed  to  monitor  all  6  student  stations.  The 
TET  raised  the  question,  "Can  effective  training 
be  conducted  with  a  1:6  ratio?"  By  comparing 
delays  and  performance  results  of  2  groups  of 
students,  it  was  found  that  students  trained  using 
the  1:6  ratio  experienced  as  few  or  fewer  delays 
and  performed  as  well  or  better  than  the  students 
taught  at  the  1:3  ratio. 


Scenario  Generating  Program 

The  Scenario  Generating  Program  is  the 
primary  support  software  package  which  provides 
the  scenario  creator  with  the  capability  to  gen¬ 
erate  and  modify  scenario  data.  This  program  was 
designed  to  be  used  by  inexperienced  instructor 
personnel.  Knowledge  of  programming  languages  and 
special  skills  is  therefore  not  required.  The 
Scenario  Generating  Program  contains  graphic 
representations  of  the  locations  of  switches, 
circuit  breakers,  and  indicators  which  relate  to 
the  actual  equipment.  Data  to  be  entered  is  in 
logical  terminology.  For  example,  the  scenario 
creator  may  want  to  illuminate  or  extinguish  an 
indicator  on  the  student  station.  This  is  accom¬ 
plished  by  simply  entering  the  word  "ON"  or  "OFF" 
in  the  field  next  to  the  graphic  representation  of 
the  specific  indicator.  Figure  10  illustrates 
these  graphic  representations. 


Scenario  Data  Base  Structure 

An  important  factor  which  adds  to  the  effi¬ 
ciency  of  the  scenario  data  base  generation  is  the 
reuseability  of  step  elements.  Before  the  reuse- 
ability  concept  is  addressed,  the  hierarchial 
structure  of  a  scenario  must  be  covered. 

One  exercise  contains  a  variable  number  of 
frames,  with  each  frame  containing  a  variable 
number  of  steps.  Therefore,  there  is  a  three 
level  structure  inherent  in  scenario  topography. 
Each  of  these  elements  (exercise,  frame  and  step) 
is  assigned  a  unique  element  number  and  is  stored 
in  a  specific  memory  location.  The  scenario 
elements  can  be  recalled  and  modified.  Exercises 
can  borrow  frames  or  steps  from  another  exercise, 
therefore,  there  is  complete  reuseability  of 
elements  at  the  lower  two  levels. 

This  technique  reduces  system  storage  require 
ment,  because  borrowed  elements  do  not  have  to  be 
stored  twice.  Savings  are  also  realized  using  the 
pivot  concept,  because  the  scenario  versions 
derived  from  the  master  only  have  to  store  new 
step  elements,  not  the  common  ones.  Figure  11 
shows  the  savings  realized  by  this  reuseability 
design  feature. 

The  importance  of  this  design  feature  is  that 
once  a  training  objective  is  coded  into  a  portion 
of  a  scenario,  that  portion  can  be  lifted  and 
incorporated  into  any  other  scenario  that  includes 


Figure  10.  Scenario  Generating  Program  Display 

313 


T 


EXERCISE  24 


/ 


EXERCISE  39 


\ 


\ 


step  200 
step  201 
step  204 
step  210 


step  201 
step  204 
step  216 


step  201 
step  202 
step  210 
step  216 
step  217 


-  elements  processed  in  the  2  exercises 

-  Unique  elements  (memory  locations) 

-  SAVINGS  in  memory  =  9 

T9~ 


Figure  11.  Savings  Realized  by  Data  Base 
Reuseabil ity 


the  same  training  objective.  As  an  example,  one 
of  the  objectives  of  a  beginning  hostile  scenario 
is  to  train  the  operator  in  the  use  of  the  TACFIRE 
prompts.  A  group  of  frames  was  generated  to  meet 
this  objective.  In  the  more  advanced  hostile 
scenarios,  wherever  the  same  objective  was  reiter¬ 
ated  for  training  reinforcement,  it  was  only  neces¬ 
sary  to  copy  the  TACFIRE  frames  and  integrate  them 
into  the  other  scenarios.  In  this  way,  the 
instructor  personnel  need  not  waste  time  reinvent¬ 
ing  the  vrfieel  as  would  have  to  be  done  with  a  more 
traditional  software  approach. 

Configuration  Management 

The  single  most  important  tool  in  the  manage¬ 
ment  of  scenario  generation  activities  is  the 
faithful  adherence  to  configuration  control  guide¬ 
lines.  Because  elements  are  assigned  a  unique 
element  number,  an  indexing  system  is  required  to 
keep  track  of  elements  in  use.  The  authoring 
language  integral  to  the  Firefinder  trainer  has 
this  built  in  directory  for  accounting  purposes. 
Program  management  requires  visibility  of  the 
expansion  potential  of  the  data  base.  A  configur¬ 
ation  utility  also  exists  vrftich  prints  out  all 
elements  not  in  use. 

Because  elements  are  reuseable,  many  exercises 
might  be  affected  by  the  modification  of  an  ele¬ 
ment.  A  cross  reference  program  has  been  developed 
to  determine  which  higher  level  elements  contain  a 
particular  lower  level  element.  Before  any  modifi¬ 
cation  of  existing  elements  is  performed,  a  cross 
reference  is  run  to  determine  the  total  impact  of 
the  proposed  change  throughout  the  scenario  data 
base. 


During  the  generation  of  a  scenario,  the 
paper  documentation  listing  is  changed  often. 
Configuration  utility  programs  can  print  an  entire 


exercise  Including  all  frames  and  steps  in  a  logi¬ 
cal  sequence  by  sequence  breakdown.  This  same 
utility  can  print  all  the  steps  of  the  data  base 
in  numerical  fashion  for  program  package  delivery 
to  the  customer. 

Maintenance  Training 

Organizational  maintenance  tasks  taught  by  the 
trainer  emphasize  removing  and  replacing  circuit 
cards.  Card  cage  assemblies  are  represented  by 
full  sized  photographic  panels.  Instead  of 
removing  and  replacing  the  faulty  card,  the 
student  enters  a  unique  3  character  remove  and 
replace  (RSR)  code  which  can  be  located  on  the 
photopanel  above  the  card  to  be  removed.  (Refer 
to  figure  12)  The  scenario  can  then  grade  the 
student's  entry  for  correctness  and  eliminate  the 
fault  indications  from  the  student's  stimuli,  if 
the  R  S  R  code  entry  is  correct.  The  fault 
indications  will  remain  if  an  Incorrect  card  is 
exchanged. 

In  some  cases  the  organizational  maintenance 
technician  needs  to  align  a  circuit  card  assembly 
using  a  potentiometer  adjustment.  For  these 
cases,  the  trainer  has  provided  the  real  pots  and 
software  emulation  to  simulate  the  normal  system 
response  to  the  pots  being  turned. 

By  customizing  the  maintenance  trainer  to 
simulate  what  is  only  needed  for  organizational 
level  training,  a  cost-effective  approach  is 
realized.  All  hardware  assemblies  that  were 
needed  for  particular  training  tasks  were 
designed  into  the  system.  No  extra  expense  was 
incurred  by  designing  simulated  circuit  cards, 
which  would  never  be  needed  for  this  level  of 
maintenance  training. 

Trainer  Diagnostics 


To  aid  the  instructors  and  trainer  mainte¬ 
nance  personnel  in  isolating  hardware  faults  in 
the  trainer,  two  diagnostic  scenarios  were 
provided.  These  scenarios  perform  an  analysis  on 
both  the  operator  and  maintenance  portions  of  the 
student  station  and  allow  support  personnel  the 
ability  to  isolate  faults  down  to  the  indicator, 
switch,  or  circuit  card  assembly.  Using  these 
diagnostic  scenarios,  the  mean-time-to-repair  of 
less  than  30  minutes  has  been  demonstrated. 

Generic  Simulation 


An  additional  customer  benefit  is  that  the 
trainer  has  the  generic  ability  to  simulate  other 
systems  with  a  similar  hardware  configuration.  By 
relabeling  the  switches  on  the  present  equipment 
panels,  scenarios  can  be  generated  to  train  the 
student  on  tasks  unrelated  to  the  original  design 
concept.  As  an  example,  a  scenario  to  train 
operators  for  the  Army's  automated  surveying 
system,  "Positioning  Azimuth  Determining  System" 
(PADS)  was  designed  using  the  Firefinder  student 
station.  No  software  changes  were  needed,  and 
therefore  the  cost  for  this  scenario  was  very 
minimal.  More  elaborate  training  conversions  are 
feasible  by  adding  additional  equipment  panels  and 
software  support  modules  to  the  existing  trainer. 
In  this  way,  unrelated  training  environments  can 
co-exist  within  the  same  trainer  without  the  need 
for  separate  computers,  instructor  stations,  or 
peripheral  equipment. 
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SUMMARY  ACKNOWLEDGEMENT 


The  Flrefinder  Trainer  incorporates  a 
scenario  data  package  which  translates  each  of  the 
training  requirements  into  an  interactive  system 
between  the  student  and  the  computer.  The  trainer 
is  basically  a  simulator,  which  emulates  each  of 
the  functions  of  the  prime  system  with  a  high 
degree  of  fidelity.  The  scenarios  then  enhance 
this  simulation  through  the  use  of  CAI/CMI  tech¬ 
niques  to  transform  the  system  into  an  effective 
training  device.  This  approach  was  validated  by 
the  training  effectiveness  tests  conducted.  The 
scenario  data  base  is  easily  managed  because  of 
the  various  configuration  utility  programs 
provided.  The  Chief  Programmer  concept  and  the 
family  breakdown  maximize  the  scenario  writing 
staff's  efficiency  in  developing  the  training 
programs.  The  further  breakdown  of  scenarios  into 
3  difficulty  levels  optimizes  the  Interface 
between  the  student  and  the  instructor.  The 
features  of  the  maintenance  trainer  allow  the 
student  to  Increase  his  proficiency  in  repairing 
the  system  to  a  very  high  degree.  The  Scenario 
Generator  Program  and  design  of  the  scenario  forms 
allow  the  creators  to  develop  a  training  package 
rapidly  and  In  a  cost-effective  manner.  In  the 
same  way,  retro-fitting  the  trainer  to  Incorporate 
changes  in  the  prime  system  Is  simplified.  These 
and  other  design  features  allow  the  customer  to 
realize  substantial  savings  throughout  the  life 
cycle  of  the  device. 


The  authors  wish  to  acknowledge  the  con¬ 
tributions  of  Major  Robert  White  of  PM-TRADE, 
Flrefinder  Training  Devices  Program  Manager,  In 
directing  the  development  of  the  scenario  data 
base  concepts  described  in  this  paper. 
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ABSTRACT 


Simulators  are  now  generally  accepted  and  even  sought  to  support  maintenance  training; 
especially  troubleshooting.  This  paper  deals  with  the  problems  that  are  encountered  by 
relying  solely  on  a  task  analysis  for  establishing  simulator  design  specifications  and  how 
these  problems  can  be  avoided  through  the  use  of  a  functional  engineering  analysis  of  the 
prime  equipment.  It  explains  how  a  functional  engineering  analysis  can  alleviate  the 
problems  of  determining  how  much  front-end  analysis  Is  necessary  for  simulator  design, 
aspects  of  the  configuration,  such  as,  three-dimensional  versus  two-dimensional  simulators, 
and  the  depth  of  the  cost  analysis.  The  necessity  of  conducting  a  functional  engineering 
analysis  In  order  to  prepare  viable  simulator  design  specifications  Is  elaborated  with  an 
example.  This  necessarily  leads  to  a  description  of  the  Increasing  role  of  a  new  breed  of 
engineer  in  designing  simulators  for  maintenance  training. 


INTRODUCTION 


In  recent  years,  simulators  for  mainten¬ 
ance  training  have  come  Into  vogue.  The  argu¬ 
ment  now  Is  not  over  their  value  and  how  well 
they  perform  compared  to  actual  equipment 
trainers.  Now  the  arguments  are  whether  sim¬ 
ulators  should  be  three-dimensional,  two-dimen¬ 
sional  (l.e.  level  of  physical  fidelity),  what 
types  of  display  media  should  be  used,  what 
levels  of  functional  fidelity  are  necessary, 
and  there  are  attempts  being  made  to  establish 
and  standardize  procurement  procedures.  There 
Is  discussion  and  research  regarding  the  extent 
of  student  cueing  and  tracking,  and  how  simu¬ 
lators  can  support  different  cognitive  styles. 
There  Is  concern  about  user  acceptance.  And 
the  problem  being  given  the  most  attention  Is 
how  much  front-end  analysis  Is  necessary  In 
order  to  establish  and  specify  simulator  design. 
The  task  analysis  that  occurs  during  an  Instruc¬ 
tional  System  Development  process  Is  generally 
thought  to  be  adequate  as  a  means  to  specify 
simulator  design.  We  have  found  through  the 
actual  experience  of  building  In  excess  of  500 
simulations  that  a  new  concept  must  be  applied 
In  all  maintenance  simulator  design;  that  Is  a 
functional  engineering  analysis.  This  In  turn 
Implies  the  need  for  a  new  breed  of  engineering 
talent  In  the  training  community. 


PROBLEMS/BACKGROUND 


Obviously  there  are  problems  and  questions 
to  be  answered,  but  pitfalls?  "An  unsuspected 
difficulty,  danger,  or  error  that  one  may  fall 
Into,"  according  to  Webster.  With  the  many 
questions  and  unresolved  problems  regarding  the 
design  of  maintenance  training  simulators  there 
are  obviously  pitfalls.  However,  I  refer  here 


to  an  area  of  effort  and  investigation  which 
has  not  as  of  yet  received  any  great  emphasis. 
And  that  is  the  lack  of  attention  given  the 
prime  equipment  during  the  Instructional  System 
Development  (ISO)  effort.  There  Is  little  to 
no  regard  given  to  a  functional  engineering 
analysis  of  the  prime  equipment  as  it  relates 
to  Its  own  operation  and  how  that  may  affect 
training  or  the  specification  and  procurement 
of  maintenance  training  simulators.  (It  is  not 
n\y  Intention  to  debate  the  value  of  simulation 
devices  for  maintenance  training,  but  for  this 
discussfon,  to  accept  it  as  a  viable  training 
tool ) . 

The  pitfall  occurs  through  a  lack  of  the 
realization  that  the  prime  equipment  must  be 
characterized  In  Its  own  right  before  a  legiti¬ 
mate  simulator  specification  can  be  prepared. 
This  must  be  done  by  the  procuring  agency  prior 
to  RFP  release  or  provided  for  In  the  solicita¬ 
tion  as  the  first  task  to  be  undertaken  by  the 
successful  bidder. 

Too  often  maintenance  simulators  are 
viewed  as  being  the  baby  brothers  (or  sisters) 
of  flight  simulators  and  therefore  need  not  be 
designed  with  the  considerations  that  go  Into 
flight  simulator  design.  For  example,  the 
flight  characteristics  analysis  Is  analogous  to 
an  engineering  analysis  of  the  hardware  from  a 
maintenance  aspect. 

The  problems  of  designing,  specifying  and 
constructing  maintenance  training  simulators 
may  Indeed  be  more  difficult  to  handle  than  the 
problems  affecting  the  design  of  flight  simula¬ 
tors.  The  maintenance  activity  occurs  within 
and  without  the  prime  Item  In  flight  simula¬ 
tion  there  Is  at  least  the  convenience  of 
knowing  that  the  student  pilot’s  activity  has  a 
nice  neat,  well  defined  set  of  parameters  all 
within  the  cockpit.  In  maintenance  training. 
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the  activity  parameter  has  the  potential  for 
being  the  entire  aircraft,  and  similarly  for 
whatever  the  prime  item  might  be.  If  It’s  a 
radar,  the  maintenance  simulation  must  contain 
both  operator  functions  and  intrusions  into  the 
device  itself. 

Capt.  D.H.  Westbrock,  Commanding  Officer  of 
the  Naval  Training  Equipment  Center,  writing  in 
the  December  6,  1979  issue  of  the  NTEC  Center- 
line  recounts  a  meeting  regarding  acquisition 
and  support  Issues  prior  to  the  first  Inter- 
service/Industry  Training  Equipment  Conference 
in  Orlando  last  November,  between  representa¬ 
tives  of  Industry  and  the  Service's  training 
organizations.  That  meeting,  "was  held  in  an 
attempt  to  get  some  agreement  on  the  problems 
facing  our  business  and  to  explore  possible 
actions  for  Improvement.  It  is  worthy  to  note 
that  there  was  little  to  no  disagreement  on  the 
problems  nor  the  actions  for  improvement." 

He  recounts  that  the  problem  areas  can  be 
generally  summarized  as  the  lack  of  proper 
front-end  analysis;  lack  of  early  open  discus¬ 
sion  with  Industry  on  requirements  which  result 
In  proposals  and  contracts  without  a  clear 
understanding  of  the  requirement;  contract 
performance,  especially  schedule  adherence  Is 
basically  unsatisfactory. 

I  would  add  to  that  list  a  specific  aspect 
of  front-end  analysis.  And  that  is  the  lack  of 
consideration  given  to  the  prime  equipment  for 
which  a  proposed  training  program  and/or  a 
maintenance  simulator  Is  to  be  designed;  espe¬ 
cially  the  lack  of  attention  given  to  the  engin¬ 
eering  design  concept  of  the  prime  equipment. 

It  is  a  commonly  held  belief  and  is  gener¬ 
ally  true  that  there  Is  lack  of  proper  front-end 
analysis.  We  as  contractors  often  respond  to 
RFP's  and  "bend  metal"  before  we  have  or  are 
given  the  opportunity  to  clearly  identify  the 
training  objectives,  or  before  it  has  been 
determined  how  the  device  will  fit  into  a 
proposed  or  existing  training  system. 

This  problem  can  be  tempered  if  we  take  a 
good  look  at  what  enables  the  design  of  a  simu¬ 
lator  and  what  does  not.  The  training  community 
believes  that  it  answers  the  training  simulator 
requirements  with  its  task  analysis.  Indeed, 
the  task  analysis  specifies,  if  done  properly, 
what  the  maintenance  training  simulator  should 
do.  It  does  not  provide  specifications  for  how 
the  simulator  will  do  what  it  is  expected  to  do. 
Just  as  In  the  design  of  a  flight  simulator, 
there  Is  a  considerable  gap  between  specifying 
what  the  simulator  must  provide  for  a  student 
and  making  It  happen. 

Generally,  when  most  people  speak  of  front- 
end  analysis  they  are  referring  to  an  approach 
to  course  design  which  Is  commensurate  with  the 
INTERSERVICE  PROCEDURES  FOR  INSTRUCTIONAL  SYSTEM 
DEVELOPMENT  (IPISD)  or  Instructional  Systems 
Development  (ISO)  as  it  Is  typically  known. 
These  procedures  spell  out  In  great  detail  how 
to  objectively  determine  the  training  require¬ 
ments  necessary  to  bring  a  person  to  some  speci¬ 


fied  level  of  maintenance  capability.  They  pay 
homage  to  the  selection  of  training  aids, 
usually  presuming  that  there  are  off-the-shelf 
types  of  equipment  available;  namely  closed 
circuit  TV,  video  tape,  audio  visual  materials 
of  various  sorts,  programed  instruction 
materials,  both  CAI  and  programmed  text  and 
recently,  maintenance  simulators.  However, 
there  isn't  a  hard  and  fast,  solid  procedure 
given  on  which  anyone  can  stake  very  much 
during  the  selection  process.  It  requires  a 
great  deal  of  interpretation  and  subjectivity 
based  on  personal  experience  and  notion  as  to 
what  these  training  aids  and  devices  will 
provide.  When  maintenance  simulation  Is 
desired,  there  is  even  less  Glrection  given  to 
simulator  selection,  that  is,  what  fidelity, 
functional  or  actual,  three-dimensional  or 
two-dimensional,  etc. 

A  task  analysis  as  It  relates  to  mainten¬ 
ance  simulator  design  is  somewhat  Incomplete. 
First,  consider  an  existing  piece  of  equipment 
that  has  already  been  fielded,  where  does  the 
training  analyst  go  for  his  information. 
Usually  the  tech  manual.  And  everyone  knows 
what  he  gets  from  that,  usually  outdated  infor¬ 
mation  because  the  system  has  been  upgraded 
and/or  the  tech  manual  was  written  to  reflect 
the  first  model  of  whatever  is  being  analyzed. 
The  training  analyst  is  locked  into  using  this 
material  for  his  task  analysis  and  he  may  or 
may  not  have  the  opportunity  to  use  question¬ 
naires  or  interviews  to  determine  the  actual 
work  requirements. 

If  it  is  a  prime  item  system  under  devel¬ 
opment,  then  there  is  constant  change  in  what¬ 
ever  technical  documentation  is  available  and 
no  chance  for  interviews  and  questionnaires. 

In  either  case,  the  training  analyst  is 
already  a  bit  short  on  specific  detail  regarding 
the  job,  let  alone  a  simulator. 

This  isn't  intended  to  shortchange  or  to 
downplay  the  need  for  a  legitimate  task  analy¬ 
sis,  but  rather  highlight  some  of  the  real 
problems  that  result  from  using  that  informa¬ 
tion  for  maintenance  trainer  simulation  design. 

Presume  that  a  task  analysis  has  been  done 
and  is  a  realistic  and  legitimate  description 
of  the  job  requirements.  Even  under  such  cir¬ 
cumstances  it  is  impossible,  but  not  believed 
to  be  so  by  many,  to  design  a  simulator  without 
actually  going  to  the  prime  equipment  and 
performing  a  functional  engineering  analysis. 
Fortunately,  there  is  an  awareness  of  this 
deficiency  beginning  to  permeate  the  training 
coninunlty. 

A  recent  RFP  issued  by  the  Air  Force  for 
“The  Development  of  a  General  Purpose  Trainer... 
for  Flight  Simulator  Troubleshooting  Training"1  ' 
used  a  task  analysis  to  determine  that  trouble¬ 
shooting  was  the  primary  activity  required 
by  maintenance  technicians  for  two  particular 
MOS  ratings.  Who  would  not  agree  and  take 
as  a  normal  course  of  events,  the  fact 
that  any  maintenance  activity  is  for  the 
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ultimate  objective  of  keeping  something  running 
or  If  It  Isn't  running  or  not  running  right, 
fixing  It  and  making  It  run  as  It  should.  In 
any  case,  when  we  consider  the  highest  order 
objective  of  a  maintenance  technician,  it's  just 
that,  keeping  something  running  as  It  should  and 
thus  requires  some  form  of  diagnosis. 

The  very  significant  aspect  of  that  RFP  Is 
that  "experience  has  shown  that  troubleshooting 
requires  an  derail  understanding  of  the  system 
logic  and  the  flow  of  control  actions  within  the 
system."  Also,  "prior  experimental  studies  have 
shown  that  top  down  learning  of  a  complex  system 
is  an  effective  approach.  This  approach  is 
consistent  with  the  whole-part-whole  concept  of 
instruction.  The  student  first  learns  the 
mission  or  goal  of  the  device  and  the  operation 
of  each  of  the  subsystems  that  are  used  to 
achieve  that  mission.  Individual  parts,  then, 
are  seen  as  contributors  to  the  overall  system 
objectives.  The  system  and  its  operation  Is 
taught  and  understood  at  the  macrolevel  before 
the  student  is  ever  taught  abcut  individual 
system  components."  Continuing  from  the  back¬ 
ground  given  In  that  RFP.  "Initial  trouble¬ 
shooting  training  can  also  be  taught  at  the 
macrolevel,  and  students  can  learn  to  rapidly 
identify  the  major  subsystems  which  may  be  In 
error  for  a  given  malfunction  condition.  Addi¬ 
tional  training  may  be  given  to  provide  a  more 
detailed  understanding  of  each  subsystem.  When 
training  reaches  the  level  of  the  smallest 
replaceable  unit.  It  need  go  no  further:  Addi¬ 
tional  training  would  not  be  truly  job  related." 
It  adds  "component  parts,  then  are  always  taught 
within  the  context  of  total  system  operation  and 
not  as  isolated  things  unto  themselves."  It  is 
about  time  that  this  type  of  thinking  has  begun 
to  show  itself  In  various  procurements. 

Here  Is  a  case  where  the  instructional 
concept  Is  being  specifically  included  in  the 
development  of  the  training  and  also  formulating 
the  trainer/simulator  design.  The  RFP  also 
calls  for  a  functional  analysis  of  the  prime 
equipment  In  order  to  achieve  this.  "A  central 
feature  required  of  the  Troubleshooting  Trainer 
is  the  description  and  presentation  of  the 
system  to  be  maintained  In  terms  of  functional 
flow  logic.  That  is,  the  system  hardware  is 
analyzed  Into  a  limited  number  of  functional 
systems,  each  system  having  a  specific  goal  to 
be  achieved  with  respect  to  the  total  system 
operation.  These  functional  systems  are  further 
subdivided  into  a  limited  number  of  subsystems 
so  that  the  complete  digital  flight  simulator 
system  can  be  represented  by  a  "logic  tree" 
arrangement  of  subsystems  to  the  level  of  detail 
required  to  provide  the  required  training.  The 
functional  goal  or  objective  of  each  subsystem 
Is  succinctly  described,  and  these  objectives 
are  documented  In  terms  of  Input-output  re¬ 
lationships.  This  method  of  system  description 
enhances  the  instructional  capability  of  this 
trainer,  and  teaches  system  logic  directly." 
This  approach  to  maintenance  simulator  design 
will  Insure  the  acquisition  of  cost  effective 
and  training  effective  simulators. 


"A  series  of  training  systems  called  the 
F-16  Simulated  Aircraft  Maintenance  Trainers 
(SAMT),  built  by  Honeywell  for  the  General 
Dynamics  corporation,  are  being  evaluated  by 
the  U.S.  Air  Force  at  Hill  AFB,  Odgen,  Utah. 
There  are  ten  different  trainers  In  the  series, 
each  designed  to  simulate  a  specialized  sub¬ 
system  of  the  F-16."  (2) 

The  RFP  that  solicited  these  training 
simulations  was  nothing  like  the  one  Just 
referenced.  It  required  an  elaborate  training 
system  with  a  set  of  simulation  panels,  an 
Instructor  station  Master  Simulation  Control 
Console  providing  data  displays  and  Inputs, 
setting  the  training  situation,  student 
communications,  random  access  slide  projectors, 
and  the  system  computer.  The  RFP  specification 
went  to  great  length  to  specify  the  program 
constructs  and  formats. 

One  of  the  Simulation  Panel  Sets  was 
required  to  simulate  over  600  Tech  Orders  and 
some  800  malfunctions.  This  kind  of  thinking 
Implies  that  the  student  must  be  exposed  to 
every  malfunction  he  will  encounter  on  the  job. 
What's  he  gonna  do  when  he  gets  there  —  take 
the  simulator? 

Pity  the  guy  trying  to  build  these  without 
obtaining  the  Information  available  only  through 
a  functional  engineering  analysis.  No  wonder 
critics  of  maintenance  simulators  say  they  are 
prone  to  failure  and  they  cost  too  much.  This 
sort  of  thing  will  confirm  their  criticisms. 
And  now  there  Is  an  Air  Force  DRFP  looking  for 
a  similar  set-up  for  the  impending  F-15  SAMT 
procurement. 

This  approach  to  simulator  procurement 
Includes  the  operation  of  the  instructional 
process  and  presumes  the  Instructional  concept. 
It  preempts  any  significant  enhancements  or 
constraints  that  may  be  obtained  from  a  task 
analysis  and  thorough  prime  equipment  analysis. 

Because  "simulators  are  now  known"  to 
affect  positive  training  results,  training  Is 
expected  to  be  effectively  enhanced  by  the  use 
of  the  ultimate  maintenance  simulator  without 
considering  the  ramification  on  cost  and  deli¬ 
ver. 

The  first  RFP^  referenced  Is  rare  In 
that  It  explicitly  requires  a  functional  analy¬ 
sis  of  the  prime  equipment.  A  functional 
analysis  of  the  prime  equipment  Is  usually 
taken  for  granted  If  It's  ever  considered. 
There  is  not  the  attention  paid  to  It  that 
there  Is  to  the  analysis  of  training  objec¬ 
tives,  job  behavioral  descriptions  and  so  on. 
Why?  As  mentioned  earlier,  it  Is  presumed  to 
be  understood  and  to  be  contained  within  the 
task  analysis,  or  there  Is  not  one  within  the 
procuring  agency  qualified  to  do  It,  so  it  Is 
Ignored.  But  how  can  the  simulator  emulate 
something  If  what  It  Is  to  emulate  Is  not 
thoroughly  defined. 
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To  reiterate,  there  Is  a  large  gap  between 
the  ISD  task  analysis  results  and  the  actual 
building  of  a  maintenance  training  simulator. 
Too  many  people  within  the  training  design  and 
analysis  community  and  In  the  training  schools 
believe  that  with  the  completion  of  the  task 
analysis,  the  simulation  specification  Is  com¬ 
plete.  It  Is  hardly  started.  At  most  It  will 
provide  Information  about  what  may  be  expected 
of  the  simulator. 

Obviously,  the  point  being  made  Is  that  a 
functional  engineering  analysis  of  the  prime 
equipment  to  be  simulated  must  be  carried  out 
either  formally  or  Informally,  If  a  viable 
simulator  Is  to  be  constructed. 


DISCUSSION-FUNCTIONAL  ENGINEERING  ANALYSIS 


A  functional  engineering  analysis  of  the 
prime  equipment  Is  an  analysis  of  that  equipment 
with  respect  to  its  operation,  normal  and  mal¬ 
functioning  as  that  operation  pertains  to  the 
level  of  maintenance  which  Is  required  by  the 
specified  maintenance  activity  and  the  training 
concept  and  methods  Intended  to  be  employed. 
The  level  of  maintenance  generally  being  Organ¬ 
izational  maintenance  (0-level)  or  Intermediate 
maintenance  ( I- level ) .  Where  0-level  Involves 
line  replaceable  units  and  I-level  involves 
component  repair.  The  functional  engineering 
analysis  requires  the  engineering  analyst  to 
document  the  logical  operation  of  the  prime 
equipment  In  terms  of  Its  functions  between  and 
among  Its  subsystems. 

The  task  analysis  can  bound  what  it  Is  the 
simulator  will  have  to  do,  but  until  a  func¬ 
tional  analysis  of  the  prime  equipment  is  com¬ 
pleted,  it  doesn't  provide  the  means  or  all  the 
decision  making  capability  necessary  to  deter¬ 
mine  what  the  simulation  device  will  contain. 

In  fact,  the  functional  engineering  analy¬ 
sis  of  the  prime  equipment  will  provide  inputs 
to  the  task  list.  If  the  functional  analysis  Is 
done  correctly,  it  will  determine  every  sub- 
assembly  or  component  that  can  be  replaced, 
repaired,  adjusted,  serviced,  etc.  at  the  given 
level  of  maintenance  and  prescribed  level  of 
simulation  (0-Level  and  I-Level).  The  level  of 
maintenance  prescribes  the  level  of  access  that 
must  be  available  In  the  simulation  for  the 
potential  user  (the  student).  It  sets  up  para¬ 
meters  for  determining  what  level  of  Intrusion 
Into  the  prime  equipment  must  be  considered  by 
the  engineering  analyst.  In  other  words,  it 
establishes  the  depth  to  which  the  technician 
can  go  In  his  maintenance  activity.  As  a  result 
of  that.  It  will  prescribe  test  equipment 
required  and  Instrumentation  on  the  prime  equip¬ 
ment  and  test  points  that  must  be  available  to 
the  student  on  the  simulator.  In  doing  this 
analysis  then,  the  engineer  must  locate  and 
determine  all  replaceable,  repairable,  adjust¬ 
able,  and  serviceable  components  and  sub- 
assemblys  In  the  system,  and  how  they  affect 
one  another. 


The  result  of  this  analysis  may  not  seem 
to  be  that  much  different  from  what  the  task 
analysis  should  describe.  But  the  functional 
analysis  Is  basically  thing  or  equipment 
oriented  whereas  the  task  analysis  is  person 
or  student  oriented.  And  like  it  or  not,  if 
someone  Is  going  to  design  a  maintenance 
training  simulator  then  he  has  to  be  able  to 
describe  the  simulator  in  terms  of  the  thing 
to  be  simulated.  The  functional  analysis  will 
uncover  every  conceivable  test  point  and  its 
response  (Including  visual  inspections,  auditory 
cues.  Instrumentations,  etc.). 

The  engineering  analyst,  in  his  logical 
documentation,  uncovers  maintenance  techniques, 
whether  known  or  not  by  "panels  of  experts"  or 
subject  matter  experts.  This  occurs  when  he 
describes  the  system  in  terms  of  sets  of  logi¬ 
cal  operating  algorithms.  It  will  describe  all 
tests,  test  points  or  activities  within  and  on 
the  prime  equipment  that  are  necessary  to 
perform  any  of  the  maintenance,  and  particu¬ 
larly  the  troubleshooting  activities,  and  at 
the  same  time  reflect  the  response  of  the  prime 
equipment  over  time.  (We  recently  completed 
the  design  and  production  of  a  number  of  main¬ 
tenance  simulations  for  a  major  transport 
aircraft.  The  functional  engineering  analysis 
provided  the  contractor  with  data  that  led  to 
the  redesign  of  his  prime  equipment,  and  also 
uncovered  numerous  discrepancies  in  the  main¬ 
tenance  manuals  necessitating  a  substantial 
document  revision.) 

When  a  functional  engineering  analysis  is 
completed,  a  mathematically  logical  model  can 
be  written  or  programmed  so  that  it  accepts  any 
legitimate  input  at  any  of  the  given  input 
ports  and  provides  the  proper  output  at  any 
selected  output  port  during  any  legitimate 
operating  condition,  on  a  continuous,  dynamic 
basis;  including  both  abnormal  responses  during 
a  simulated  malfunction  and  normal  conditions; 
and  If  desired,  allow  a  normal  system  configur¬ 
ation  to  be  degraded. 

The  functional  engineering  analysis  pro¬ 
vides  the  means  to  develop  the  core  program. 
When  this  Is  completed,  and  is  done  appropri¬ 
ately,  any  of  the  training  requirements  can  be 
added  to  the  program.  In  other  words,  if  a 
programmed  model  is  developed  with  respect  to 
the  prime  equipment  for  the  simulation,  then 
peripheral  program  can  be  added  to  that  core 
program  as  required.  This  would  include  simu¬ 
lator  operations  necessary  to  satisfy  the 
training  approach;  other  capabilities  such  as 
monitoring  student  actions,  cues,  use  of  a  CRT, 
printers,  etc.  It  also  provides  for  an  easy 
upgrade  of  the  simulation  as  modifications  are 
made  to  the  prime  system.  And  most  signifi¬ 
cantly,  the  display  or  presentation  medium  can 
be  two-dimensional,  three-dimensional,  a  TV 
screen,  paper  and  pencil;  whatever  Is  cost 
effective  and  training  effective. 

The  task  analysis  can  be  used  to  determine 
how  much  of  all  the  particulars  uncovered  in 
the  engineering  analysis  are  necessary  to  be 


320 


! 


built  Into  the  simulation  device  for  It  to  fit 
the  prescribed  training  requirements.  The 
training  approach  will  govern  how  the  presenta¬ 
tions  will  be  made,  and  If  there  Is  enough 
feedback  and  crosstalk  between  the  simulation 
designer  and  the  training  analyst,  a  viable 
model  can  be  developed  which  will  enable  the 
construction  of  a  simulator  which  can  satisfy 
the  prescribed  training  methodology.  However, 
there  must  be  communication  between  the  training 
analyst  and  the  engineering  analyst;  better 
still  if  the  training  analyst  Is  an  engineer. 
Uhat  I'm  saying  Is  that  It's  time  to  recognize 
the  necessity  of  bringing  the  engineering 
analyst  directly  Into  the  mainstream  of  training 
analysis.  Not  "Interdisciplinary  communication" 
through  a  manufactured  Interface  of  forms, 
procedures,  and  processes,  but  actually  part 
of  the  training  community.  In  effect,  create 
a  new  engineering  job  function. 

If  a  program  model  accurately  represents 
the  simulation  desired,  whether  It  Is  a  func¬ 
tional  logic  representation  of  the  system,  or 
actual  component  or  black  box  test  points,  that 
simulation  can  be  used  In  whatever  way  the 
training  objectives  require;  even  when  a  large 
system  Is  broken  down  Into  so  called  part  task 
trainers.  The  functional  engineering  analysis 
will  determine  how  each  subsystem  (part  task 
trainer)  relates  to  the  whole  system.  These 
part  task  trainers  will  be  subsets  of  the  major 
system  components.  If  the  prime  equipment 
analysis  Is  accurate,  it  will  allow  any  type  of 
simulation  configuration. 

However,  for  a  simulator  to  be  effective, 
there  must  be  a  sharing  of  the  engineering 
constraints  regarding  the  simulation  Itself,  the 
prime  equipment  particulars  and  the  training 
requirements.  And  the  only  way  the  simulation 
design  can  be  effectively  discussed  Is  to  pro¬ 
vide  enough  time  for  the  engineering  analyst  to 
thoroughly  understand  and  specify  the  prime 
system  requirements,  and  then  Interrelate  with 
the  training  analyst.  This  Is  not  now  being 
overtly  done. 

Since  there  Is  seldom  enough  resource 
allocated  or  available  to  conduct  a  thorough 
front-end  analysis,  and  since  a  functional 
engineering  analysis  Is  a  necessity  In  building 
a  simulator.  It  can  be  used  as  the  stepping  off 
point  for  a  front-end  analysis  or  to  augment  the 
front-end  analysis.  Without  the  functional 
engineering  analysis  whether  formally  prescribed 
or  not,  there  Is  no  way  an  objective  cost 
analysis  of  a  simulator  design  can  be  made. 

To  help  make  clear  what  a  functional  engin¬ 
eering  analysis  Is,  an  example  Is  provided. 


EXAMPLE 

Consider  a  training  requirement  for  the 
maintenance  of  the  electrical  system  on  a  spark 
Ignition  engine.  The  training  objectives  might 
be  as  follows.  (Understand  that  these  are  only 
for  example  and  for  the  case  of  brevity,  only 
highlighted).  A  spark  Ignition  engine  Is  used 
here  as  an  example  because  It  Is  hoped  that  It 


will  be  easily  understood  by  most  people.  The 
higher  order  training  objectives  could  appear 
something  like  this. 

1)  Perform  prescribed  routine  preventive 
maintenance  functions. 

2)  Perform  functional  checkout  tests  of 
the  engine  electrical  system  according 
to  prescribed  technical  manual  proced¬ 
ures  to  ensure  that  engine  operation 
meets  specified  operating  standards. 

3)  Perform  repair  functions  as  may  be 
required  as  a  result  of  the  system 
falling  functional  checkout  tests 
and/or  other  unscheduled  (trouble¬ 
shooting)  maintenance  as  required. 
(Diagnose  faults.) 

The  next  order  would  Include  the  following, 
or  similar  objectives; 

1)  Use  the  prescribed  technical  manual  to 
follow  procedures. 

a)  Locate  proper  tables,  schematics, 
etc.  (be  able  to  use  tech  manual 
as  required  to  perform  and  meet 
job  function  requirements.) 

2)  Select  and  use  proper  and  appropriate 
tools. 

3)  Select  and  properly  use  necessary  and 
appropriate  support  test  equipment. 

4)  Follow  divisional  maintenance  procedures 
for  prescribed  maintenance  activities. 

5)  Perform  listed  troubleshooting  (or 
diagnostic)  activities.  For  this 
example  consider  only  the  electrical 
system  of  this  hypothetical  engine. 
Malfunctions  will  be  discussed  later. 
(There  are  problems  specifying  trouble¬ 
shooting  activities  to  the  same  detail 
that  procedures  can  be  specified. 
Symptoms  can  be  specified,  but  the  same 
symptom  can  be  caused  by  different 
component  failures.  Failures  provide 
different  symptoms  at  different  points 
of  intrusion  Into  a  system.  Trouble¬ 
shooting  trees  fix  the  diagnostic  paths 
available  and  become  overly  cumbersome 
as  simulation  specifications.) 

A  listing  of  objectives,  would  also  Include 
such  Items  as  tracing  wires  using  electrical 
schematic  diagrams,  using  test  equipment  such 
as  ohmmeter/voltmeter,  ampmeter,  oscilloscope, 
low  voltage  circuit  testers.  Identification  of 
parts,  location  of  parts,  etc. 

By  Its  nature  the  functional  engineering 
analysis  will  Include  a  malfunction  analysis. 
The  malfunction  analysis  Is  often  catagorlzed 
as  a  separate  activity.  But  In  the  process  of 
specifying  a  maintenance  simulator  that  Is  to 
be  used  for  troubleshooting,  It  must  occur 
within  the  functional  engineering  analysis  In 
order  to  specify  parameters  for  the  simulation. 
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The  task  analysis  would  also  state  what 
malfunctions  are  repaired  during  the  maintenance 
activity  based  on  frequency  of  occurrence, 
criticality.  Information  from  the  Logistics 
Support  Analysis  (LSA),  and  so  forth.  It  would 
also  estimate  the  difficulty  of  fault  tracing 
based  on  practicing  technician  feedback.  The 
functional  engineering  analysis  would  also  use 
the  LSA,  but  In  Its  own  right  would  Identify  all 
replaceable,  repairable,  serviceable,  adjustable 
Items.  Because  of  Its  functional  aspect,  It 
would  uncover  causes  and  their  effects;  failed 
components  and  the  malfunction  symptom.  The 
functional  engineering  analysis  would  provide 
the  means  to  select  malfunctions  for  simulation 
which  affect  a  large  portion  of  the  system  and 
hence,  require  more  of  the  system  to  be  Investi¬ 
gated  In  order  to  determine  the  cause.  In  this 
way  the  student  could  thoroughly  exercise  the 
system,  and  the  requirements  for  hundreds  of 
malfunctions  be  eliminated. 

Stating  the  malfunction  symptom  does  not 
necessarily  state  Its  cause  or  reflect  a  grouping 
or  classification  of  malfunction  causes.  For 
Instance,  our  engine  may  have  as  one  of  Its 
malfunctions  "engine  does  not  start".  This 
could  be  caused  by  a  myriad  of  problems.  The 
major  systems  at  fault  could  be  the  Ignition, 
the  fuel,  the  charging,  or  the  cranking  system. 
Then  in  each  system,  any  separate  component 
could  be  at  fault,  with  each  different  falling 
component  giving  off  various  and  different 
symptoms  during  the  test  and  diagnostic  phase. 
(See  Table  2  for  a  sampling  of  malfunctions.) 

Presume  for  this  example  that  among  the 
training  aids  and  devices  deemed  necessary  for 
the  Instruction  there  Is  a  simulator  requirement 
and  the  simulation  need  has  been  defined  as  the 
electrical  system  of  this  hypothetical  engine 
for  a  respective  MOS. 

How  Is  a  simulator  specified,  and  by  whom. 

In  order  to  have  someone  build  It?  The  job  task 
analysis  or  the  training  objectives  specify  what 
it  should  do.  Right?  Wrong!  They  specify  what 
the  student  will  do  In  training,  on  the  simula¬ 
tor  and  hopefully,  be  able  to  do  when  he  eventu¬ 
ally  gets  to  his  job. 

Here  Is  where  the  functional  engineering 
analysis  becomes  critical,  but  not  at  the  point 
at  which  It  should  come  into  play.  It  should 
have  been  started  much  earlier.  In  conjunction 
with  the  task  analysis  portion  of  the  ISD  effort. 
Even  If  no  simulator  Is  contemplated,  a  func¬ 
tional  engineering  analysis  is  necessary  If 
reasonable  courseware  Is  to  be  prepared. 

The  functional  engineering  analysis  should 
be  done  to  satisfy  the  level  of  maintenance 
prescribed  for  the  simulator  training  applica¬ 
tions,  and  the  depth  of  Intrusion  during  mainten¬ 
ance  activity  Is  defined  by  the  job  activity. 

The  maintenance  activity  Is  predetermined  In  the 
LSA  or  In  the  engineering  design  concept,  and 
states  what  kind  of  maintenance  can  be  per¬ 
formed,  how  It  can  be  performed,  and  provides  a 
means  for  determining  who  will  and  can  do  It 
(the  MOS);  0-Level,  or  I-Level  maintenance. 


It  Is  Interesting  to  note  that  the  Army's 
Skill  Performance  Aids  (SPAS)  concept  requires 
an  activity  similar  to  the  functional  engin¬ 
eering  analysis  described  here.  The  first  step 
In  the  SPAS  development  process  Is  the  conduct 
of  a  front-end  analysis  which  defines  perform¬ 
ance  requirements  through  equipment  analysis 
and  functional  analysis  which  In  turn  yields  a 
total  task  list. 

Continuing  with  the  engine  example,  the 
benefits  that  can  be  derived  from  a  functional 
engineering  analysis  will  become  clear.  Once 
the  level  of  maintenance  is  determined,  the 
device  Intrusion  Is  also  specified.  The  engin¬ 
eering  analyst  can  state  explicitly  the  func¬ 
tional  and  logical  Interrelationship  of  the 
device  at  the  component  or  subsystem  level  that 
Is  relevant  to  the  level  of  maintenance.  He 
needs  to  go  one  level  deeper  than  the  level  of 
maintenance  to  analyze  causes  and  effects.  In 
our  engine  example,  the  task  analysis  has 
specified  normal  operation  and  malfunctions  are 
Inferred  by  the  troubleshooting  requirement. 
The  engineering  analysis  now  determines  those 
parts  of  the  device  that  must  be  assessed  to 
determine  normal  operation  and  their  appear¬ 
ance,  both  normal  and  abnormal.  When  the 
malfunctions  are  considered,  the  functional 
engineering  analysis  becomes  even  more  critical 
As  mentioned  earlier,  the  same  symptom  can  be 
observed  and  yet  be  caused  by  various  and 
different  malfunctioning  parts  or  components. 
Considering  the  malfunction  symptom  "the  engine 
will  not  start."  This  could  have  many  causes. 
(See  Table  2.) 

The  engineering  analysis  breaks  the  prime 
equipment,  the  engine  and  In  particular  Its 
electrical  system.  Into  Its  design  subsystems; 
with  this  engine:  the  charging  system,  the 
Ignition  system,  the  cranking  system,  the 
fuel  system,  and  the  mechanical  system.  We  are 
concerned  here  only  with  the  electrical  systems 
(charging,  cranking,  and  Ignition). 

It  provides  data  such  as  that  shown  In 
Table  1.  The  normal  configuration  would  be 
described  first  with  consideration  being  given 
to  evaluating  the  system  l.e.  the  test  points, 
visual  Inspections,  Instrument  probes,  auditory 
signals  and  so  forth  that  will  be  used.  The 
functional  engineering  analysis  must  also  state 
the  relation  of  one  subsystem  to  another  and 
the  effects  of  each  one  upon  all  others  It 
affects. 

This  listing  would  continue  In  similar 
detail  for  each  of  the  other  systems  and 
their  components.  The  reason  this  analysis 
must  go  to  the  component  level,  or  one  deeper 
than  the  maintenance  activity,  is  that  when 
different  components  fall  within  the  subsystem, 
they  can  produce  different  symptoms  and  differ¬ 
ent  responses  within  the  system.  For  example, 
no  voltage  from  the  alternator  could  be  caused 
by  the  voltage  regulator  or  the  diodes  within 
the  alternator  and  the  first  noticeable  oper¬ 
ating  symptom  would  be  a  discharged  battery. 
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TABLE  1 

SUBSYSTEM  COMPONENTS 


Comoonent/System 
1.0  Charging  System 


1.1.0  Alternator 


Conditions 


Functions/Use 


Normal  Maintains  battery  electrical 

charge 


Abnormal  Overcharges  battery. 

Does  not  maintain  charge. 


Normal  Provides  current  to  maintain 

battery  charge;  output  varies 
with  engine  speed,  controlled 
by  Voltage  Regulator. 


Abnormal 
Open  Field 
Shorted  Field 
Open  diodes  ^ 
Brushes  open 
Stator  shorted^ 


No  output  or  too  high  an  output. 


1.1.1.  Electrical  Connections 

Normal 

Provides  specified  output. 

Field 

Grounded 

Battery  (output) 

Open 

Too  low  or  no  output. 

Ground 

Corroded 

1 

. 

1.2.0  Voltage  Regulator 

Normal 

Controls  voltage  level 
supplied  to  battery  by 
regulating  alternator 
field  voltage. 

0  -  1200  engine  RPM 
voltage  output  varies 

0  -  14.2  V. 

1200  -  max  RPM 

14.2  V 

Abnormal 

Field  voltage: 

too  high”! 
too  low  J 

e— 1 

Alternator  overheats 

(may  short-out). 

Battery  discharges. 

L. 

1.2.1  Electrical  Connectors 

Normal 

No  voltage  frm  alternator, 

Abnormal 

No  voltage  to  battery. 
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Additional  Information  must  be  obtained 
regarding  the  prime  equipment  -  the  prime 
equipment  response  and  how  the  response  can 
be  observed.  For  the  hypothetical  engine  It 
would  look  like  the  following: 


Activity  Equipment  Response 

Crank  Engine  Engine  can  crank:  normal 

slow 

intermittent 
not  at  all 


The  response  can  be  observed  on  an  auditory  basis,  or  with  a 
low  scale  tachometer,  and  a  determination  made  as  to  how  the 
simulator  will  provide  that  response. 


Activity  Equipment  Response 

Test  alternator  output  None  on  engine  -  Ampmeter,  VOM,  or 

Generator  Output  tester  needed. 


The  response  can  be  observed  on  the  face  of  the  test  Instrument 
with  the  Instrument  set-up  In  a  prescribed  manner.  The  set-up 
must  be  specified. 


Next  needed  are  the  malfunctions  that  are 
to  be  simulated.  These  can  be  overlayed  onto 
the  normal  operating  conditions  during  program¬ 
ming.  With  a  functional  engineering  analysis 
It  can  be  determined  which  malfunctions  affect 
the  greatest  part  of  the  system  and  coupled  with 
a  frequency  of  occurrence  analysis  and  critical¬ 
ity  those  that  provide  the  most  training  bene¬ 
fit  can  be  selected.  (An  example  of  what  happens 
when  this  Is  not  done  is  a  D.O.D  specification 
calling  for  623  malfunctions.) 

Table  2  Is  a  sampling  of  an  adequate  list 
of  malfunctions  which  will  provide  training  in 
using  the  test  equipment  and  practice  diagnosing 
the  charging  and  cranking  systems  of  this  hypo¬ 
thetical  engine. 


For  each  Item  In  the  malfunction  list  there 
will  of  necessity  be  additional  Information 
provided.  At  some  point  the  effect  of  each 
fault  on  all  systems  being  represented  In  the 
simulation  must  be  stated.  Not  shown  In  the 
table,  but  a  very  real  situation  Is  the  compound 
malfunction,  and  where  the  symptom  Is  placed  for 


use  on  the  simulator.  Consider  malfunction 
number  13,  Its  symptom  could  have  been  "engine 
will  not  start"  with  the  same  fault  and  remedy 
as  listed.  In  this  case,  the  battery  would  be 
discharged  and  not  able  to  crank  the  engine 
because  the  alternator  Is  not  able  to  charge  the 
battery.  Symptom  number  13  relates  to  the 
subsystem  —  the  alternator. 

If  the  alternator  In  the  charging  system 
falls  (malfunction  number  12)  and  the  battery  Is 
fully  charged,  the  engine  could  still  be  started 
but  only  a  certain  number  of  times  -  until  the 
battery  was  discharged  to  the  point  where  It 
could  no  longer  supply  enough  power  to  the 
cranking  motor  to  turn  the  engine,  but  the 
engine  could  continue  to  run  because  less  power 
is  required  to  keep  the  engine  running  than  Is 
required  to  crank  It  for  starting. 

The  engineering  analysis  would  determine 
how  long  the  engine  could  run  at  various  speeds 
after  one  start  on  a  fully  charged  battery,  and 
after  being  started,  run,  stopped  and  started 
again  until  the  battery  was  discharged.  This 
could  be  detailed  with  a  set  of  algorithms: 


1 
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TABLE  2 


MALFUNCTION  LIST 

SYMPTOM 

FAULT 

REMEDY 

1. 

Engine  will  not  start 

Dirty  positive  battery  post 
connection  (looks  dirty) 

Clean  and  tighten 
battery  post 
connection. 

2. 

Engine  will  not  start 

Discharged  battery  (most 
cells  have  low  specific 
gravl ty) 

Replace  battery. 

3. 

Engine  will  not  start 

Faulty  Ignition  switch 
(open  starter  terminal) 

Replace  Ignition 
switch. 

4. 

Engine  will  not  start 

Faulty  solenoid  (open 
wl ndings) 

Replace  solenoid. 

5. 

Engine  will  not  start 

Badly  pitted  solenoid 
disc  (open) 

Replace  disc. 

6. 

Sometimes  the  engine 
cranks  slow  and  Is 
hard  to  start 

Pitted  solenoid  disc 
(high  resistance) 

Replace  disc. 

7. 

Engine  Is  hard  to 
start 

High  resistance  In 
positive  battery  cable 
(not  visible) 

Replace  battery 
cable. 

8. 

Engine  will  not  Start 

Faulty  battery  cable, 
loose  connection  at 
starter 

Clean  and  tighten 
battery  cable 
connector. 

9. 

Sometimes  the  engine 
cranks  slow  and  Is 
hard  to  start 

Starter  motor  amperage 
draw  too  high 
(high  resistance  In 
windings) 

Replace  starter 
motor. 

10. 

Sometimes  the  engine 
cranks  slow  and  1$ 
hard  to  start 

Poor  connection  between 
negative  battery  cable 
and  engine  block 

Clean  and  tighten 
connection. 

11. 

Engine  cranks  slow 
and  is  hard  to 
start 

Faulty  regulator 
(voltage  regulator 
needs  adjustment) 

Adjust  regulator. 

12. 

No  charging  action 

Faulty  alternator 
(brush  broken) 

Replace  alternator. 

13. 

Sometimes  the  engine 
cranks  slow  and  Is 

Faulty  alternator 
(burned  out  diodes) 

Replace  alternator. 

hard  to  start 
(Battery  voltage 
low) 


325 


Run  time  ■  (state  of  charge  -  previous  run  time 

x  running  charge 
loss  rate) 

-  (no.  of  previous 
starts  x  start 
charge  loss) 

State  of  charge  •  available  amps 
Running  charge  loss  rate  ■  Amps  loss  x  Engine  RPM 

Start  charge  loss  ■  X  amps  per  start 

These  algorithms  would  obviously  have  to  be 
more  specific  and  quantified.  Once  that  Is  done 
though,  they  can  easily  be  programmed. 

The  functional  engineering  analysis  can  be 
used  as  a  tool  to  determine  the  simulator  struc¬ 
ture  by  providing  a  detailed  listing  of  the  prime 
equipment  responses.  These  can  In  turn  be  used 
to  aid  a  cost  trade-off  analysis  during  the 
design  phase.  For  example: 

Equipment  Response  - 

Engine  cranks:  at  normal  speed 

intermittently 

Observation  - 


The  simulator  can  be  provided  with  a  syn¬ 
thesizer  to  produce  the  sound  or  a  written 
message  describing  the  Interpretation  of  the 
sound.  Meter  readings  can  be  shown  on 
simulated  meters,  a  graphics  display,  a  slide 
visual,  or  a  written  message. 

Without  a  functional  engineering  analysis, 
there  Is  no  other  source  that  will  provide 
accurate  descriptions  of  what  Is  to  be  sim¬ 
ulated. 

As  far  as  a  three-dimensional  simulator 
structure  is  concerned,  the  functional  engin¬ 
eering  analysis  provides  hard  data  that  can  be 
used  to  avoid  falling  Into  the  trap  of  speci¬ 
fying  a  three-dimensional  device.  Presuming  the 
simulator  In  question  is  going  to  be  used  as  a 
means  to  acquire  troubleshooting  skills,  and 
not  necessarily  motor  skills,  then  simulating 
the  prime  equipment  responses  with  a  three- 
dimensional  device  would  require  an  actual 
equipment  trainer  (AET)  (and  no  simulator),  and 
poor  troubleshooting  practice.  If  a  mix  of 
two-dimensional/three-dimensional  components  Is 
required,  determining  what  dimensional  combina¬ 
tions  are  feasible  can  only  be  done  with  data 
obtained  from  an  engineering  analysis. 


SUMMARY 


The  functional  engineering  analysis  pro¬ 
vides  the  Intimate  knowledge  of  the  prime 
equipment  necessary  to  construct  an  effective, 
affordable  maintenance  training  simulator. 
With  data  available  from  a  functional  engin¬ 
eering  analysis,  decisions  can  be  made  on  how 


much  functional  fidelity  Is  necessary  or  Is 
worth  providing.  An  example  was  given  of  an 
engine  with  a  good  battery,  an  Inoperative 
alternator  and  the  resultant  variable  run  and 
start  combinations.  The  cost  In  analysis  and 
subsequent  programming  must  be  traded-off 
against  the  training  benefit  that  would  result 
from  Incorporating  that  kind  of  fidelity.  The 
same  concept  could  be  provided  to  the  student 
by  judicious  selection  of  malfunctions  or 
programmed  exercises  which  highlight  that 
concept. 

Conducting  an  effective  functional  engin¬ 
eering  analysis  for  a  maintenance  training 
simulator  requires  a  person  with  rare  talents; 
an  engineer  who  uses  his  engineering  and  techni¬ 
cal  experience  and  expertise  as  an  engineer  and 
as  a  training  analyst.  He  must  act  In  both 
capacities  at  the  same  time.  He  must  extrapo¬ 
late  from  a  prime  equipment  what  Is  necessary 
to  produce  a  viable  training  device. 

A  functional  engineering  analysis 
answer  the  questions  of  what  kind  of  stu  i 
cueing  Is  required  In  a  simulator  or  how  oest 
to  accommodate  cognitive  styles,  etc.  but  It 
will  provide  a  solid  data  base  for  establishing 
the  operational  characteristics  of  a  simulator 
for  maintenance  training.  It  will  provide  the 
data  necessary  for  a  contractor  to  accurately 
define  his  costs.  And  it  enables  realistic 
schedules  to  be  developed  and  adhered  to. 
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ABSTRACT 


As  Automatic  Test  Equipment  (ATE)  for  Avionics  becomes  increasingly  costly  it  becomes  prohibitive  to  provide  enough 
test  stations  to  allow  adequate  individual  hands-on  training  for  maintenance  personnel.  The  LT-2  Trainer  for  F-16 
Aircraft  Immediate  Shop  (AIS)  helps  alleviate  this  problem.  The  LT-2  trainer  consists  of  a  mini-computer  system  identical 
to  that  of  the  full  F-16  AIS  stations  without  the  actual  stimulus  and  measurement  hardware.  The  trainer  is  thus  about 
one  tenth  of  the  cost  and  complexity  of  the  full  test  station.  An  Innovative  Software  Package  allows  test  data  captured 
on  a  (till  test  station  to  be  played  beck  on  the  trainer.  This  provides  the  student  with  a  realistic  simulation  of  actual 
test  runs.  All  the  keyboard  inputs  and  CRT  responses  are  identical  to  those  of  a  full  test  station.  A  Library  of  test 
programs  and  captured  data  sets  allows  both  test  station  Maintenance  Programs  and  Avionics  Test  Programs  to  be 
simulated.  The  captured  data  sets  can  be  for  good  or  faulty  test  runs.  Faults  can  also  be  inserted  into  good  data  sets 
to  artificially  create  simulated  test  failures.  The  techniques  used  on  the  LT-2  trainer  allow  cost-effective  individual 
hands-on  training  for  complex  ATE. 


BACKGROUND 


The  LT-2  Maintenance  Trainer  concept  was 
developed  by  General  Dynamics  Electronics  to  provide 
a  low-cost  trainer/simulator  for  the  F-16  Aircraft 
Intermediate  Shop  (AIS)  Automatic  Test  Equipment  (ATE). 
The  F-16  AIS  consists  of  four  independent  automatic 
test  stations.  A  typical  station  is  shown  in  Figure  1 
with  the  control  system  components  common  to 
all  four  types  indicated  with  numbers.  Each  of  these 
test  stations  is  designed  to  test  a  specific  group 
of  F-16  Line  Replaceable  Units  (LRUs).  The  four 
test  stations  and  their  associated  primary  LRU  testing 
areas  are  as  follows: 

1.  Test  Station  -  Flight  Control  Computer, 

Fire  Control  Computer,  Inertial  Navigation  System. 

2.  Displays/Indicator  Test  Station  -  Heads- 
Up  Display  and  Radar  and  Navigation  Indicators. 

3.  Procesaors/Pneumatics  Test  Station  -  Air 
Data  Computer,  Radar  Computer,  Radar  Digital 
Signal  Processor. 

4.  Radio  Frequency  Test  Station  -  Radar  and 

ILS. 

The  F-18  AIS  test  stations  are  considered  third 
generation  automatic  test  equipment.  Tests  are 
controlled  by  stored  programs  written  in  the  ATLAS 
test  language.  Common  Stimulus  and  Measurement 
functions  are  provided  in  the  Stimulus  Unit  and  Mea¬ 
surement  Subsystem  (SUMSS).  Peculiar  stimulus 
and  measurement  functions  are  provided  in  Tester 
Replaceable  Units  (TRUs).  Previous  generations 
of  ATE  consisted  entirely  of  rack  and  stack  TRUs. 


INDEX 

NO 

DESCRIPTION 

1 

COMPUTER  HP  21 

2 

DISC  DRIVE.  HP  2905 A 

3 

DISC  CONTROLLER  HP  13037A 

4 

I/O  EXTENOER.  HP  12979A 

5 

STATION  PRINTER.-HP  9866A 

6 

STATION  CONTROL  PANEL.  2113100-001 

7 

CRT/KEY80ARD  BEEHIVE  B500  4000  0010 

Figure  1.  F-16  AIS  Test  Station 

The  USAF  initially  approached  maintenance  and 
operations  training  on  the  F-16  AIS  using  the  classic 
method  of  instruction:  that  is,  the  use  of  an  instructor 
in  front  of  a  class  teaching  from  adapted  engineer!^ 
data,  followed  up  by  limited  hands-on  exposure  to 
the  actual  equipment.  This  approach  was  later  aug¬ 
mented  through  the  use  of  video  taped  presentations. 
Students  are  taught  the  theory  of  operation  of  both 
the  test  stations  and  LRUs. 
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The  usual  constraints  were  and  are  operational:  lim¬ 
ited  time  and  money,  limited  availability  of  test 
stations  and  LRUs,  limited  data,  and  changing  design. 
There  is  very  little  time  for  extensive  hands-on  train¬ 
ing.  There  is  practically  no  exposure  to  faulted 
or  faulty  equipment.  Consequently,  the  student 
sees  or  experiences  only  the  results  of  testing  opera¬ 
tional  equipment.  When  exposed  to  non-operational 
conditions,  he  is  not  permitted  to  find  the  fault, 
but  is  relegated  to  over-the-shoulder  observation 
of  the  trouble  shooting  procedure.  The  methods 
used  dictate  a  heavy  reliance  on  downstream  on- 
the-job  training  to  achieve  proficiency.  The  result 
of  the  training  is  that  the  student  can  perform  only 
the  rudimentary  operations  of  LRU  testing  and  ATE 
self-test.  The  limited  exposure  to  the  physical  equip¬ 
ment  permits  the  student  to  Remove  and  Replace 
(R&R)  components  from  either  the  test  station  or 
LRU  when  directed  by  software  controlled  messages 
but  he  is  not  prepared  for  extensive  trouble  shooting. 


be  improbable  that  succeeding  student  classes  could 
"memorize"  the  available  fault  exercises.  Realism 
in  the  trainer  control  hardware  was  also  desired,  i.e., 
operational  keyboard/CRT  and  station  control  paneL 
The  USAF  also  wanted  to  achieve  the  "ideal"  training 
scenario  of  having  the  student  operate  the  "ATE" 
within  the  first  day  of  training.  Finally,  the  trainer 
must  provide  a  way  of  giving  the  student  maximum 
hands-on  exposure  at  a  cost  far  less  than  that  associated 
with  providing  a  full  A IS  station  for  each  student 

A  fortunate  coincidence  of  hardware  and  software 
design  in  the  F-16  AIS  provided  the  means  to  achieve 
most  of  the  requested  trainer  reqiarements  by  util¬ 
izing  the  control  system  hardware  from  the  AIS  and 
specially  developed  software  to  simulate  the  remainder 
of  the  AIS  test  station.  This  is  the  LT-2  trainer  concept. 


LEVELS  OF  TRAINING 


TRAINER  REQUIREMENTS 


In  mid-1978,  the  USAF  suggested  that  an  AIS 
trainer  could  be  a  viable  approach  to  solving  some 
of  their  training  problems.  The  conceptual  design 
process,  which  ultimately  resulted  in  the  LT-2  F-16 
AIS  Trainer,  proceeded  along  these  lines.  Since 
the  F-16  AIS  is  a  software  intensive  system,  provisions 
for  training  the  student  in  the  operational  commands 
and  the  use  of  the  various  built-in  software  trouble¬ 
shooting  tools  must  be  made.  The  trainer  would 
have  to  permit  the  student  to  have  some  control 
over  his  hands-on  training  experience  such  that  he 
could  repeat  specific  training  exercises  or  phases 
if  needed  or  desired.  It  would  have  to  address  the 
typical  three  levels  of  training  associated  with  any 
ATE  training  program: 

1.  Operational  training 

a.  Operate  ATE  (Self  Test  Mode) 

b.  Operate  the  ATE  (LRU  or  Interface 
Test  Adapter  (ITA)  Test  Mode) 

2.  Removal  and  Replacement  of  faulty  compo¬ 
nents  when  directed  by  software  controlled  messages. 

a.  ATE 

b.  LRU  AT  A 

3.  Identification  and  isolation  of  faults  not 
handled  by  automatic  fault  isolation. 

A  USAF  concern  throughout  the  early  phases 
of  the  F-16  AIS  program  was  the  maintenance  of 
the  Computer  Subsystem.  This  concern  dictated 
that  the  trainer  have  the  capability  to  train  students 
in  the  care  and  feeding  of  the  computer  and  its  periph¬ 
erals.  The  USAF  wanted  a  trainer  which  used 
the  actual  software  resident  on  the  test  station  and 
the  actual  software  which  is  used  to  test  the  LRUs, 
or  at  least  as  realistic  a  representation  of  the  effects 
of  that  software  as  possible.  They  also  expressed 
the  desire  to  have  capability  for  a  large  number 
of  inser table  faults,  Urge  enough  so  that  it  would 


Level  One  Trainin 


All  AIS  test  stations  utilize  the  same  control  system 
which  consists  of  an  HP  21  MX  computer  and  associated 
peripherals  including  a  disc  drive,  CRT,  keyboard 
thermal  printer,  and  a  station  control  paneL  During 
actual  AIS  station  operation,  the  operator  inserts 
a  disc  containing  the  LRU  or  self -test  program,  connects 
the  LRU  (if  applicable)  and  via  the  keyboard  end  station 
control  paneL  directs  the  stored  LRU  or  self -test 
program.  Test  results  and  operator  messages  are 
received  on  the  CRT,  station  control  panel  and  a 
thermal  printer.  The  student  can  experience  all  of 
these  level  one  operational  exercises  on  the  LT-2 
trainer  with  the  exception  of  physically  connecting 
the  LRU. 


Level  Two  Trainin 


The  level  two  training  (removal  and  repUcement 
of  faulty  components  as  directed  by  software  con¬ 
trolled  messages)  can  be  fully  experienced  only  for 
the  computer  subsystem.  For  the  baUnce  of  AIS 
test  hardware  and  for  LRU  failures,  all  of  the  operator 
controls  and  messages  can  be  realistically  experienced 
up  to  the  point  of  actually  repUcing  the  faulty  hard¬ 
ware.  The  limitation  is  a  physical  one,  since  the  trainer 
as  now  configured  is  a  2D+  trainer;  that  is,  the  student/ 
machine  interaction  is  limited  to  the  CRT  dispUy 
and  keyboard  and  the  station  control  paneL  The  (+) 
indicates  that  for  the  Computer  Subsystem  there 
is  full  three-dimensional  training  capability. 


Level  Three  Trainin 


Identification  and  isolation  of  faults  not  properly 
identified  by  the  AIS  self -test  or  LRU  diagnostic 
software  can  be  simulated  on  the  trainer  and  all  the 
avaiUble  software  troubleshooting  tools  and  technique* 
can  be  taught  with  the  limitation  again  of  not  being 
able  to  physically  handle  the  faulty  hardware.  In 
this  case,  an  instructor  would  evaluate  the  suitability 
of  the  student's  approach  to  such  a  problem  and  the 
student's  analysis  of  the  data  available  from  the  trouble¬ 
shooting  tools.  The  trainer  software  can  then  be 
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directed  to  yield  a  good  or  faulty  test  result  on  the 
next  pass,  indicating  the  success  of  failure  of  the 
diagnosis  and  proposed  corrective  action. 


SYSTEM  AND  SOFTWARE  DESIGN 


Common  Control  Bus 

The  AIS  test  station  computer  controls  all  stimu¬ 
lus  and  measurement  hardware  via  a  common  control 
bus.  The  computer  acts  as  the  bus  controller,  sending 
digitally  coded  commands  to  and  requesting  data 
from  all  of  the  addressable  hardware  modules  con¬ 
nected  to  the  bus. 


Once  the  data  capture/data  playback  approach 
had  been  proven,  it  was  found  that  either  test  station 
hardware  failures  or  LRU  hardware  failures  could  be 
simulated  by  modifying  the  digital  data  which  is  input 
from  the  test  station  hardware.  This  failure  simulation 
capability  is  implemented  using  a  fault  insertion  table. 
This  table  allows  the  user  (instructor)  to  modify  any 
data  word  read  in  from  the  station  test  hardware,  thus 
providing  an  almost  unlimited  fault  insertion  capability. 
Each  fault  insertion  table  entry  contains  the  ATLAS 
statement  number  at  which  the  data  modification  is 
to  occur,  the  data  hardware  control  bus  address  and 
the  new  data  (in  octal)  along  with  a  data  mask  which 
indicates  which  bits  are  to  be  modified. 


When  a  test  is  in  progress  on  an  AIS  test  station, 
all  digital  control  data  is  transferred  between  the 
computer  and  the  test  hardware  over  this  data  bus 
system.  The  station  control  software  generates 
the  data  output  to  the  hardware  and  reacts  to  the 
data  input  from  the  hardware  and  to  operator  com¬ 
mands.  The  control  software,  in  tum,  drives  the 
operator  displays  on  the  station  control  panel  and 
CRT.  The  LT-2  system  software  stores  the  digital 
data  input  from  the  hardware  into  a  disc  file  during 
ATLAS  test  program  execution  on  an  AIS  station. 

This  data  is  read  back  on  a  trainer  and  substituted 
for  the  data  input  from  the  actual  hardware.  In 
this  way  the  LT-2  software  can  simulate  the  operation 
of  a  complete  test  station  on  a  trainer  station  that 
only  contains  a  computer,  (fisc  system  and  station 
operator  console. 


Operator  Modes 


The  trainer  software  operates  in  either  the 
data  capture  or  data  playback  mode  on  an  AIS  test 
station  and  in  the  data  playback  mode  only  on  the 
trainer  station.  In  the  data  capture  mode,  the  I/O 
routines  store  only  the  digital  data  input  from  the 
hardware  in  a  "captured  data"  (fisc  file.  The  data 
output  to  the  hardware  (control  words)  are  not  needed 
for  playback. 


The  data  capture  operation  is  performed  by 
executing  an  ATLAS  test  program  on  a  full  F-16 
test  station  with  an  LRU  or  self -test  adaptor  con¬ 
nected.  During  data  capture,  all  input  digital  data 
is  stored  in  a  disc  file  for  retrieval  in  the  data  play¬ 
back  mode.  Once  the  data  has  been  captured  for 
a  particular  ATLAS  test  program,  that  program 
can  be  executed  in  the  data  playback  mode  on  the 
trainer  station.  In  the  data  playback  mode,  the  I/O 
routines  read  the  captured  data  from  the  (fisc  file 
instead  of  inputting  the  data  from  the  hardware. 
This  process  causes  the  control  software  to  react 
precisely  as  though  the  test  hardware  were  present. 
All  station  control  console  displays  and  operator 
controls  react  the  same  as  they  (fid  during  the  data 
capture  test  run  on  the  AIS  test  station. 


The  data  capture/data  playback  technique  can 
be  implemented  on  any  computer  controlled  test 
system  that  has  a  similar  data  bus  interface  and 
common  I/O  routines  which  service  the  bus. 


Failures  can  be  simulated  in  either  the  data  capture 
or  data  playback  mode  of  operation.  In  the  data  cap¬ 
ture  mode,  the  digital  data  is  modified  as  it  is  read 
from  the  hardware  and  before  it  is  stored  in  the  cap¬ 
tured  data  file.  In  the  data  playback  mode,  the  digital 
data  is  modified  as  it  is  read  from  the  captured  data 
file,  but  the  captured  data  file  is  not  modified. 

There  are  some  restrictions  on  simulated  failures. 
The  simulated  failures  force  the  ATLAS  test  program 
through  a  specific  predetermined  path.  In  the  data 
capture  mode,  the  user  (instructor)  must  insure  that 
all  subsequent  failures  down  the  selected  path  are 
consistent  with  the  original  simulated  failure  in  order 
to  produce  a  realistic  hardware  failure  simulation. 

When  simulating  a  failure  in  the  data  playback  mode, 
the  user  (instructor)  must  be  sure  the  inserted  fault 
does  not  force  the  ATLAS  test  program  down  a  path 
which  requires  additional  hardware  data  not  present 
in  the  stored  data  file. 


When  an  ATLAS  test  program  contains  a  number 
of  program  paths  that  can  be  selected  by  the  student 
during  program  execution,  retrieving  the  proper  cap¬ 
tured  data  for  the  selected  path  can  present  a  problem. 
We  solved  this  data  retrieval  problem  using  a  branch 
table.  This  table  contains  the  first  statement  number 
of  every  program  branch  to  be  executed  and  the  disc 
file  location  (sector,  word)  of  the  data  captured  for 
that  program  branch.  The  branch  table  is  created 
by  the  user  (instructor)  and  the  file  locations  are 
filled  in  by  the  trainer  software.  Before  executing 
the  ATLAS  test  program  in  the  data  capture  mode, 
the  user  (instructor)  must  enter  the  first  statement 
number  of  all  program  branches  for  which  he  intends 
to  capture  data.  During  program  execution,  the  trainer 
software  determines  if  the  next  statement  to  be  exe¬ 
cuted  is  the  beginning  of  a  new  branch  by  comparing 
it  with  the  statement  numbers  in  the  brand)  table. 

If  the  next  statement  is  the  beginning  of  a  new  branch, 
the  trainer  software  stores  the  captured  data  file 
location  (sector,  word)  with  that  statement  number. 

The  trainer  software  can  now  locate  and  retrieve 
the  captured  data  for  each  program  branch  when 
running  in  the  data  playback  mode. 
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Lesson  Plans 

A  library  of  captured  data  sets  can  be  created 
for  LRU  test,  station  self -test,  and  station  alignment 
programs  to  provide  comprehensive  training  for  the 
student.  Generally,  one  good  test  run  and  a  number 
of  faulted  test  runs  are  collected.  The  faulted  runs 
can  be  created  by  modifying  the  input  data  using 
the  fault  insertion  table  or  by  collecting  data  sets 
running  with  bad  LRUs  or  faulty  station  hardware. 
Using  the  trainer  software,  a  technician  can  capture 
additional  data  sets  in  the  field.  Problems  encoun¬ 
tered  in  the  field  can  be  captured  on  disc  and  sent 
back  to  the  training  facility.  This  capability  can 
act  as  a  constant  feedback  loop  to  insure  that  actual 
field  problems  are  used  to  train  new  maintenance 
personnel.  Maintenance  history  will  provide  good 
LRU  candidates  for  data  capture  test  cases. 

The  instructor  can  build  lesson  plan  scenarios 
from  the  library  of  captured  data  sets.  A  number 
of  test  cases  can  be  presented  for  each  LRU  or  station 
hardware  device.  A  student  workbook  can  also  be 
provided  to  guide  the  student  to  the  proper  diagnosis 
of  hardware  problems.  The  lesson  plan  table  stored 
on  the  disc  by  the  instructor  controls  the  selection 
of  the  data  set  to  ‘  e  used  for  each  program  executed 
by  the  student  or  for  each  iteration  of  the  same 
program.  Fairly  complex  lesson  plan  scenarios  can 
be  stored,  requiring  the  student  to  execute  a  parti¬ 
cular  sequence  of  diagnostic  programs  to  locate 
a  fault  before  his  main  test  program  will  run  with 
successful  results. 

Troubleshooting  Tools 

A  number  of  trouble  shooting  tools  included 
in  the  AIS  software  can  be  utilized  on  the  LT-2  trainer 
to  assist  in  level  three  training.  One  such  tool  allows 
the  student  to  monitor  via  the  CRT  or  printer  all 
of  the  data  passing  over  the  control  bus  during  the 
execution  of  a  program.  Proper  analysis  of  this 
data  can  assist  in  pinpointing  faults  which  the  test 
programs  do  not  isolate. 


APPLICABILITY 


The  concept  of  the  LT-2  trainer  design  is  to 
use  minimum  hardware  and  sophisticated  software 
to  permit  the  student  to  experience  the  effects  of 
actual  test  runs  on  full  AIS  stations  with  all  the 
proper  operator  machine  interactions.  This  concept 
may  be  applicable  to  a  significant  range  of  other 
trainers  for  automatic  test  equipment  and  for  other 
automated  military  or  industrial  equipment.  In  order 
for  this  approach  to  be  useful,  the  system  would 
have  to  satisfy  most  of  the  following  criteria! 

1.  Computer  controlled-automated  system 

2.  Software  intensive  -  relies  heavily  on  soft¬ 
ware  for  operator  interfaces 

3.  Common  data  bus  -  system  hardware  com¬ 
mands  and  data  travel  across  a  single  party  line 
type  data  bus 


4.  Common  I/O  driver  routines  -  all  communica¬ 
tions  to  hardware  are  via  the  same  software  I/O 
subroutine 

5.  Data  storage  available  -  sufficient  data 
storage  capability  must  be  available  to  store  all 
I/O  data  for  a  full  test  run,  mission,  etc. 

6.  Limited  program  branches  -  the  operator 
selected  program  branches  must  be  limited  to  a 
reasonable  number,  such  that  it  is  feasible  to  capture 
and  store  data  for  each  branch 

7.  Non-critical  execution  times  -  program 
execution  time  in  the  data  capture  and  simulate 
mode  may  differ  from  the  normal  execution  times 
of  the  programs  due  to  the  storage  and  retrieval 
times  for  the  data 

8.  Operator  training  priority  -  the  operator 
control  and  analysis  functions,  rather  than  hardware 
handling,  must  be  the  primary  purpose  of  the  training. 


CONCLUSION 


The  LT-2  trainer  concept,  aside  from  its  rather 
minor  limitations,  does  provide  a  cost-effective  way 
to  achieve  an  adequate  hands-on  training  environment 
fo<*  complex  ATE.  Its  limitations,  primarily  in  the 
area  of  the  handling  of  physical  hardware  can  be  over¬ 
come  with  a  minimal  amount  of  supplementary  training 
on  a  full  AIS  station. 

The  unique  concepts  of  the  LT-2  trainer  could 
be  easily  extended  to  trainers  for  other  systems  which 
satisfy  the  listed  criteria  for  applicability. 


ABOUT  THE  AUTHORS 


Arthur  J.  Mason,  Principal  Engineer,  F-16  AIS 
Software,  General  Dynamics  Electronics  Division, 
MS-UCLA  1965 

Kenneth  J.  Gibson,  Senior  Software  Engineer, 
General  Dynamics  Western  Data  System  Center,  BS- 
Oklahoma  University  1969. 

Richard  J.  Bonar,  Manager  of  Off-Site  Operations, 
General  Dynamics  Electronics  Division,  MS-NM  State 
University,  1972,  USAF  Retired  -  Weapons  System 
Aoqiisition 


i 


E-3A  MAINTENANCE  PROCEDURE  SIMULATORS  -  A  NEW  BREED  OF  CATs 


David  L.  Winter,  Senior  Research  Scientist 
American  Institutes  for  Research  in  the  Behavioral  Sciences 
Bedford,  Massachusettes 

Michael  Sturm,  AWACS  Engineering  Manager 
Honeywell,  Incorporated,  Training  and  Control  Systems  Center 
West  Covina,  California 

ABSTRACT 


Computer-Aided  Trainers  (CATs)  for  mission  avionics  subsystems  on  the  E-3A  "Sentry"  aircraft 
simulate  BITE  outputs  to  support  self-paced  practice  in  flight  line  maintenance  procedures. 
They  represent  a  new  breed  of  CATs  evolved  through  application  of  the  Instructional  System 
Development  (ISO)  process  and  a  unique  synthesis  of  hardware,  software  and  "courseware"  to 
satisfy  organizational  maintenance  training  requirements.  This  paper  is  in  two  parts.  Part  I 
summarizes  the  application  of  the  ISO  process  used  to  identify  requirements  for  training 
and  training  equipment.  Part  II  concentrates  on  the  evolution  of  the  maintenance  trainers, 
emphasizing  the  system  development  using  as  a  case  study  the  E-3A  Maintenance  Procedure 
Simulator  (MPS)  for  the  AN/ASN-118  Navigation  Computer  System. 


PART  I:  SUMMARY  OF  ISD  APPLICATION 

Selection  of  CATs  as  the  most  cost  effective 
method  for  satisfying  E-3A  training  was  strongly 
influenced  by  results  of  a  1974-76  effort  to  de¬ 
fine  training  equipment  requirements  for  four 
major  groups  of  E-3A  mission  avionics  equipment 
prior  to  delivery  of  the  first  aircraft  in 
March  1977.  Table  1  lists  these  four  equipment 
groups  and  the  specialty  codes  and  titles  for  the 
four  types  of  technicians  who  must  be  trained  to 
maintain  them. 

Scope  of  ISO  Effort 

When  properly  applied,  a  joint  contractor/ 
government  ISD  effort  can  provide  a  system  acqui¬ 
sition  program  director  with  essential  information 
needed  to  define  a  cost-effective  "mix"  of  class¬ 
room,  laboratory  and  on-job  traininq.  The  E-3A 
Sentry  engineering  uivision  ( YWE )  may  be  unique, 
in  that  since  the  inception  of  the  Airborne 
Warning  and  Control  System  (AWACS)  program,  a 
separate  section  has  been  responsible  for  all 
human  factors,  training,  and  training  equipment, 
with  contract  technical  support  from  the  American 
Institutes  for  Research  (AIR),  a  nonprofit  firm 
specializing  in  applied  education  and  training 
research.  In  the  E-3A  program,  collaboration  be¬ 
tween  contractors  and  government  personnel  in  the 


ISD  process  was  achieved  by  a  series  of  working- 
level  Technical  Interchange  (TI)  meetings  during 
the  18-month  period  from  September  1974  through 
January  1976.  A  study  contract  to  develop  speci¬ 
fications  for  training  equipment  was  initiated 
by  Boeing,  the  E-3A  prime  contractor,  in  December 
1974. 

Inputs  available  to  the  government/contractor 
ISD  team  included: 

o  Draft  Training  Plan 

o  Qualitative/Quantitative  Personnel  Require¬ 
ments  Information  (QQ  PRI ) 

0  Aerospace  Ground  Equipment  (AGE)  Plan 

o  Optimum  Repair  Level  Analysis  (ORLA) 

Reports 

o  Reliability/Maintainability  Allocations, 
Analysis  and  Assessment  Reports 


TABLE  1 

.  AWACS  MISSION  AVIONICS 

EQUIPMENT 

AWACS  MISSION  AVIONICS 
EQUIPMENT 

AFSC 

AIR  FORCE  SPECIALTY  TITLE 

Data  Processing 

305X4 

Electronic  Computer  Systems 
Repairman 

Communications 

328X0 

Avionic  Communications 
Specialist 

Surveillance  Radar 

328X2 

Airborne  Early  Warning  Radar 
Specialist 

Navigation/Guidance 

328X4 

Avionic  Inertial  and  Radar 
Navigation  Systems  Specialist 
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Figure  1  shows  a  functional  flow  diagram  of 
the  training  equipment  requirements  analysis  based 
on  the  procedures  defined  in  AFP  50-58  "Handbook 
for  Designers  of  Instructional  Systems."  Repre¬ 
sentatives  of  Air  Training  Command  (ATC),  Tactical 
Air  Command  (TAC),  E-3A  SPO  personnel  and  AIR 
participated  in  TI  meetings  with  Boeing  training 
specialists  during  the  ISO  effort,  which  was 
divided  into  six  phases: 

(1)  Review  AWACS  equipment  and  AGE  to  estab¬ 
lish  maintenance  task  descriptions. 

(2)  Determine  job  performance  and  knowledge 
requirements  for  each  Air  Force  Specialty  Code. 

(3)  Establish  maintenance  training  objec¬ 
tives  for  each  Air  Force  Specialty  Code. 

(4)  Determine  training  media  requirements 
to  accomplish  training  objectives. 

(5)  Define  candidate  training  equipment 
configurations  needed  to  satisfy  media  require¬ 
ments. 

(6)  Develop  detailed  specifications  for 
preferred  training  equipment  configuration  selec¬ 
ted  for  each  Air  Force  Specialty  Code. 

The  outputs  of  the  ISO  study  effort  included 
a  comprehensive  technical  data  package  containing 
the  "Training  Equipment  Preliminary  Requirements 
Report"  and  a  set  of  draft  specifications  and  a 
set  of  budgetary  cost  estimates.  The  methods 
used  to  define  the  training  equipment  are  des¬ 
cribed  below,  using  the  AN/ASN-118  as  an  example. 

Maintenance  Task  Analysis 

Using  the  procedures  defined  in  Volume  II  of 
AFP  50-58  as  a  guide,  maintenance  tasks  were 
analyzed  independently  by  ATC  representatives  and 
the  prime  contractor.  The  team  analyzed  the 
organizational  maintenance  tasks  defined  explicit¬ 
ly  or  implicitly  in  the  QQPRI  and  AGE  Plan.  The 
Boeing  team  analyzed  intermediate  (shop)  mainte¬ 
nance  tasks  resulting  from  the  ongoing  ORLA  and 
R/M  analyses.  Results  were  consolidated  on  stand¬ 
ard  ATC  task  description  worksheets  to  produce 
a  common  baseline  for  definition  of  learning  ob¬ 
jectives. 

Definition  Of  Learning  Objectives 

Formal  learning  objectives  for  each  class  of 
related  tasks  (e.g.  flight-line  checkout  pro¬ 
cedures)  were  first  defined  independently  by  ATC 
and  Boeing,  then  consolidated  on  a  set  of  Learn¬ 
ing  Objective  Worksheets  (LOWs)  developed  by 
AIR  for  this  purpose.  Each  LOW  identified  the 
learning  objective,  the  standard  for  its  measure¬ 
ment  and  the  analyst's  judgement  regarding  the 
best  learning  media  to  support  the  achievement  of 
the  objective.  Differences  in  media  recommenda¬ 
tions  were  resolved  at  TI  meetings.  As  a  timely 
byproduct,  the  scope  and  content  of  ATC  Course 
Training  Standards  for  resident  training  courses, 
field  training  at  TAC  bases,  and  Type  I  cadre 
training  at  contractor  and  subcontractor  facili¬ 
ties  were  also  established  at  these  TI  meetings. 


Evaluation  Of  Training  Configurations 

For  each  E-3A  mission  avionics  system,  two 
or  more  "candidate"  training  equipment  configura¬ 
tions  were  identified  and  rated  on  26  different 
characteristics  grouped  in  four  categories  as  il¬ 
lustrated  in  Figure  2.  Candidates  generally 
represented  the  extremes  of  full  simulation  and 
actual  operational  equipment,  with  some  mixed 
cases  combining  computer-assisted  instruction  (CAI) 
with  mockups  and  actual  operational  end-items. 

Thus,  for  the  AN/ASN-118  system,  Candidate  A  was 
defined  as  an  Actual  Equipment  Trainer  with 
three  student  stations;  Candidate  B  was  a  Computer- 
Aided  Training  System  using  two  student  CRT 
terminals  to  guide  students  in  the  operation  of 
front  panel  mockups;  Candidate  C  was  a  mixed 
system  combining  features  of  the  first  two  config¬ 
urations. 

Tasks  assigned  to  Categories  1  and  2  were 
classified  into  the  seven  types  used  by  the  USAF 
Human  Resources  Laboratory  for  development  re¬ 
search  on  Job  Performance  Tests  for  electronic 
technicians.  For  the  third  category  of  Training 
Suitability,  each  candidate  was  evaluated  against 
qualitative  criteria  determined  by  a  training 
literature  review  to  yield  high  training  effective¬ 
ness.  The  extent  to  which  each  candidate  wou1d 
minimize  support  requirements  was  rated  in  terms  of 
the  impact  on  facilities,  computer  programming 
support,  unique  test  equipment,  and  overall  ease  of 
maintenance. 

For  each  of  the  26  traits  shown  at  the  left  in 
Figure  2,  analysts  entered  a  rating  taken  from 


the  following  five-point  scale: 

Rating  Description  Rating 

Little  or  no  capability  1 

Partially  satisfies  criteria  2 

Satisfies  most  criteria  acceptably  3 

Satisfies  all  criteria  acceptably  4 

Satisfies  all  criteria  exceptionally  5 


The  weighting  factors  shown  in  Fiqure  2  were 
based  on  a  three-point  scales  (low/moderate/high) 
for  rating  the  task  proficiency  training  suitabil¬ 
ity,  and  the  impact  on  support  requirements.  The 
products  of  each  weight  and  candidate  rating 
were  entered  in  the  "adjusted"  column,  then  di¬ 
vided  by  the  number  of  criterion  items  to  obtain 
a  "normalized"  rating  between  1  and  5  for  each 
major  category.  Regrettably,  the  procedure  used 
did  not  include  multiple  raters  or  inter-rater 
reliability  measures.  As  indicated  in  Figure  2, 
the  Computer-Aided  Training  system  (Candidate  B) 
was  rated  highest,  and  was  therefore  recommeno.-d 
to  support  training  of  328X4  personnel  in  mainte¬ 
nance  of  the  AN/ASN-118  system. 

The  E-3A  prime  contractor's  draft  report  de¬ 
tailing  the  ratings  and  rationales  for  each  of 
the  four  recommended  training  equipment  configur¬ 
ations  was  reviewed  at  a  TI  meeting  in  September 
1975.  Results  were  curious  and  disappointing 
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to  all  concerned.  Responding  to  historical  ATC 
and  TAC  preferences  for  actual  equipment  for 
maintenance  training, the  contractor  offered  to 
supply  actual  equipment  trainers  at  an  esti¬ 
mated  cost  of  approximately  $35  million.  This 
sum  did  not  include  costs  of  support  equipment 
required  to  meet  reliability/maintainability 
specifications.  By  January  1976,  when  costs  for 
support  equipment  were  included,  the  proposed 
costs  for  actual  equipment  trainers  had  ballooned 
to  approximately  $50  million.  This  sum  greatly 
exceeded  the  funds  programmed  for  maintenance 
training  equipment  for  AWACS  mission  avionics. 

E-3A  SPO  Cost/Benefit  Analysis 

Faced  with  this  situation,  human  factors  and 
training  specialists  in  the  SPO  prepared,  with 
AIR  assistance,  a  Cost/Benefit  Analysis  showing 
the  amortized  cost  per  trainee  for  the  relatively 
small  numbers  of  trained  personnel  (11  to  70  per 
year)  required  to  man  TAC  maintenance  positions 
for  the  fleet  of  15  aircraft  then  under  contract. 
Prorated  over  a  15-year  life  cycle,  maintenance 
training  equipment  costs  would  have  ranged  from 
$21,000  to  $80,000  per  trainee.  This  study  postu¬ 
lated  and  explored  the  consequences  of  a  decision 
to  drastically  limit  MTE  procurement  to  static 
devices  and  visual  aids  for  classroom  training 
(relying  on  the  T.C.s  and  student  handouts),  de¬ 
ferring  actual  "hands-on"  practice  to  on-the- 
job  training  at  TAC  bases.  This  "devil's  advo¬ 
cate"  approach  produced  prompt  and  constructive 
responses  from  TAC  and  ATC,  generally  favoring 
a  pragmatic  mix  of  a)  selected  end-item  equipment 
withwhichATC  would  construct  its  own  communica¬ 
tion  trainer;  b)  second-shift  operations  at  TAC 
bases  to  train  computer  maintenance  technicians 
using  already-avai lable  AWACS  operator  training 
equipment;  and  c)  low-cost  front-panel  simulators 
for  the  radar  and  navigation  training  equipment. 

In  March  1976  the  E-3A  SPO  approved  this 
pragmatic  approach  to  a  cost  effective  solution  of 
training  requirements.  This  approval  was  quali¬ 
fied  by  a  recognition  that  availability  of  ade¬ 
quately  defined  procedures  in  technical  orders 
should  be  a  prerequisite  to  purchase  of  testing 
equipment.  In  May  1976,  ATC  proposed  design 
guidelines  and  preliminary  functional  requirements 
for  a  front-panel  simulator  to  support  training 
for  the  navigation  systems.  An  ATC  preference  for 
the  extensive  use  of  off-the-shelf  equipment  in 
the  simulators  and  a  competitive  procurement  for 
them  was  endorsed  by  the  E-3A  SPO  at  a  TI  meeting 
in  June  1976,  at  which  time  ATR  commenced  work 
on  the  Prime  Item  Development  Specification  (PIDS). 
Competitive  bidding  resulted  in  award  of  a  con¬ 
tract  to  Honeywell  on  30  May  1978  and  the  ASN-118 
(Tl)  simulator  for  the  E-3A  Navigation  Computer 
System  was  delivered  to  Keesler  Technical  Train¬ 
ing  Center  18  months  later  in  December  1979. 


PART  II:  THE  MAINTENANCE  PROCEDURE  SIMULATOR  (MPS) 

Trainer  design  and  development  is  a  disci¬ 
plined  symphony  made  up  of  concepts,  components 
and  people.  The  maintenance  trainer  is  a  recent 
addition  to  the  host  of,  among  others,  operator 
and  operational  trainers  for  sonar  and  avionic 
systems.  Upon  first  look  at  the  maintenance  train¬ 
er,  one  can  get  lulled  into  a  feeling  that  these 
type  of  trainers  are  relatively  simple  requiring 
less  development  rigor  and  discipline  than  pre¬ 
vious  device  types.  This  would  be  a  serious  mis¬ 
take  in  both  time  and  money. 

Maintenance  trainers  primarily  use  off-the- 
shelf  components,  well-proven  equipments,  all 
within  today's  technology.  Difficulties  in  devel¬ 
opment  schedules  and  cost  arise  because  from  past 
experience  we  typically  estimate  that  the  complex¬ 
ity  of  the  device  is  equivalent  to  the  complexity 
of  the  hardware.  Hence,  based  on  previous  exper¬ 
ience,  we  tend  to  underestimate  the  total  scope 
of  effort.  Although  the  hardware  is  straightfor¬ 
ward,  the  maintenance  trainer  is  complex  in  the 
software  and  instructional  features  domain.  The 
development  approach  for  the  maintenance  trainer 
device  must  be  carefully  conceived,  well-defined 
through  specification  and  controlled  through 
management  disciplines. 

The  E-3A  MPS  Design  Concept 

The  purpose  of  the  E-3A  MPS  is  to  supplement 
classroom  training  of  organizational  level 
maintenance  personnel  on  the  AN/ASN-118  Navigation 
Computer  System  (NCS).  The  MPS  is  designed  to 
demonstrate  operation  and  checkout  of  the  NCS. 

This  includes  the  simulation  of  malfunctions  and 
maintenance  procedures.  The  MPS  functions  as  a 
procedural  trainer  in  that  the  following  mainte¬ 
nance  can  be  performed: 

1.  Fast  System  Checkout 

2.  Complete  System  Checkout 

3.  Trouble  Analysis 

A.  Subsystem  Performance  Checks 

5.  Post  Replacement  Checkout  Procedures 

The  MPS  is  augmented  with  real-time  models  in 
order  to  support  part  task  training  on  the 
Inertial  Navigation  System  (INS)  as  well  as  the 
NCS.  A  mix  of  simulated  and  real  components 
for  various  E-3A  avionic  equipments  are  mounted 
on  flat  panels  to  provide  hands-on  manipulation 
of  equipment  during  lessons.  A  pictorial  of  the 
device  is  presented  in  Figure  3 

The  Detail  Design  Specifications 

A  formal  method  to  clarify  a  PIDS  and  document 
the  system  definition  is  necessary  for  development 
contracts.  A  first  step  towards  achieving  this 
objective  is  to  make  a  comparison  of  the  PIDS  and 
the  submitted  proposal.  The  PIDS  provides  the 
information  on  what  is  required,  the  proposal  pro¬ 
vides  information  on  what  is  priced.  Any  contra¬ 
dictions  in  the  intent  of  these  documents  should  be 
clearly  identified  as  early  as  possible  in  the 
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igure  3.  E-3A  Maintenance  Procedure  Simulator 


project  and  result  in  specification  changes. 

During  the  definition  phase,  all  other  clarifi¬ 
cations  should  also  be  prepared  and  submitted 
as  changes  to  the  PIDS.  Formal  changes  provide  a 
clear  trail  as  to  what  agreements  have  been 
reached  between  customer  and  contractor.  The  up¬ 
dated  PIDS  serves  as  a  functional  baseline  for 
detailed  product  and  software  specifications. 

A  maintenance  trainer,  such  as  the  E-3A  MPS, 
is  not  meant  to  exactly  duplicate  all  of  the 
workings  of  the  original  equipment.  The  specifc 
functions  and  malfunctions  of  the  operational 
equipment  must  be  specified  prior  to  software  de¬ 
velopment  and  fabrication. 

With  a  clearly  defined  PIDS,  plus  detailed 
maintenance  procedures,  a  Computer  Program 
Development  Specification  (Part  I  software  re¬ 
quirement  specification)  can  be  drafted.  The 
CPDS  maps  the  software  requirements  from  the  PIDS 
into  stand  alone  documents.  Care  must  be  taken 
in  generation  of  the  CPDS  to  present  requi rements, 
not  implementation.  It  is  important  not  to 
design  the  software  system  in  the  CPDS  as  this 
often  precludes  functional  group  participation 
and  can  lead  to  a  less  than  robust  software 
design. 

To  achieve  system  definition  for  the  E-3A 
MPS,  a  System  Specification  was  generated.  This 
was  an  internal  working  document,  which  de¬ 
fined  the  system  concepts  and  allocated  functions 
to  hardware,  software  and  courseware.  It  was  ac¬ 
cessible  to  the  project  team  and  communicated 
the  system  as  it  evolved  through  the  definition 
phase.  This  document  was  allowed  to  become 
obsolete  upon  conclusion  of  the  design  phase 
with  the  issuance  of  product  and  software  speci¬ 
fications. 


The  Software  System 

In  general,  software  systems  for  maintenance 
trainers  interface  with  a  wide  variety  of  periph¬ 
erals  as  well  as  panel(s)  with  multiple  switches, 
rotaries,  analogs,  keyboards  and  other  miscella¬ 
neous  components  which  represent  various  aircraft 
subsystems.  The  software  system  provides  the 
functional  capability  to  allow  for  interface  with 
the  instructor  for  lesson  preparation  and  control, 
as  well  as  student  monitoring.  The  functional 
capability  to  allow  student  interface  with  the 
trainer  must  also  be  provided. 

The  CPDS  provides  to  what  extent  the  student 
and  instructor  has  to  interface  with  the  device. 
Also  provided  is  the  degree  to  which  various  com¬ 
ponents  are  required  to  be  simulated. 

The  software  architecture  for  the  E-3A  MPS 
is  presented  in  Figure  4.  It  represents  the 
approach  Honeywell  developed  to  meet  the  E-3A 
MPS  requirements.  The  system  is  based  on  trans¬ 
posing  Technical  Orders  (T.O.s)  into  an  inter¬ 
mediate  language,  and  compiling  that  language 
to  form  a  data  base.  During  a  lesson  the  data 
base  is  used  to  provide  the  instructional 
portions  of  the  simulation. 

Specifically,  the  T.O.s  are  transposed  using 
the  Courseware  Author's  Language  (CAL).  The  CAL 
was  developed  specifically  for  this  purpose  and 


Provides  a  productive  means  of  presenting  T.O. 
information  in  a  high  order  language.  The  CAL  is 
compressed  through  use  of  the  Courseware  Author's 
Language  Generator  (CALGEN).  CALGEN  takes  the 
higher  order  language  and  transforms  it  into  a 
compressed  data  base  called  Primitive  Courseware 
Language  (PCL).  CALGEN  provides  error  diag¬ 
nostics  and  various  user  aids. 

The  application  software  is  composed  of  two 
primary  parts:  specific  trainer  software  and  the 
Procedure  Monitor.  The  specific  trainer  software 
is  composed  of  the  input/output  processors,  real¬ 
time  model  functions,  and  the  instructor/ trainee 
processors.  The  Procedure  Monitor  is  the  real¬ 
time  interpreter  which  translates  the  PCL  into 
actions  based  upon  panel  inputs. 

Honeywell  feels  that  this  architecture  is 
flexible  and  allows  for  change.  This  is  important 
in  maintenance  trainers  in  that  panel  components 
can  change  as  well  as  fidelity  requirements  over 
the  life  of  the  system. 

For  the  E-3A  MPS,  two  specific  devices,  the 
INS  and  NCS,  were  to  be  high  fidelity,  real-time 
simulations.  Defining  how  the  INS  and  NCS  worked 
to  the  extent  necessary  was  done  very  well.  The 
INS  and  NCS  work  well  enough  to  "fool"  the  student 
as  well  as  the  instructors  into  believing  that 
they  are  actual  devices.  To  give  an  example  of 
the  degree  of  fidelity,  the  INS  activated  switches 
on  depression  while  the  NCS  activated  switches  on 
release,  this  was  simulated  in  the  trainer.  In 
interfacing  these  models  with  courseware,  two  con¬ 
cepts  should  be  kept  in  mind. 

1.  Courseware/ software  interfaces  should 
be  as  simple  as  possible. 

2.  Courseware  interfaces  should  be  as 
transparent  to  the  software  as 
possible. 

These  aforementioned  items  will  aid  in  making  the 
software  and  courseware  development  simpler. 

The  Courseware 

The  courseware  starts  out  as  a  T.O.,  and 
winds  up  as  a  compressed  data  base  stored  on 
disk.  The  manner  in  which  the  courseware  evolves 
from  T.O.  to  language  is  called  T.O.  Annotation. 
The  annotation  of  T.O.s  is  a  result  of  training 
analysis  and  human  Factors  engineering.  As  a 
result  of  the  annotation,  the  T.O.  becomes  a  de¬ 
tailed  explanation  of  how  the  lesson  should 
operate. 

For  the  E-3A  MPS,  the  PIDS  specified 
that  there  should  be  varying  visual  aids  to  the 
student,  based  on  an  instructor  selected  input. 
Hence,  rules  were  laid  out  in  order  to  properly 
annotate  the  T.O.s  for  slides.  Additional 
rules  corresponded  to  maintenance  action  slides, 
set-up  slides  and  error  slides.  Besides  anno¬ 
tation,  the  T.O.s  had  to  be  transposed  into  the 
CAL.  This  was  accomplished  through  a  set  of 
rules  which  were  taught  to  the  courseware  authors 
These  rules  were  then  expanded  using  computer 
automation. 
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FIGURE  4.  E-3A  MRS  SOFTWARE  ARCHITECTURE 


The  results  of  this  approach  to  generating 
the  CAL  was  very  satisfactory.  The  273  lesson 
units  developed  for  the  E-3A  MPS  meet  their  lesson 
objectives. 


The  Hardware  System 


As  discussed  previously  hardware  systems  for 
maintenance  trainers  are  relatively  simple,  making 
use  of  off-the-shelf  equipment  as  much  as  possi¬ 
ble.  The  E-3A  MPS  is  no  exception.  The  E-3A  MPS 
is  composed  of  a  Honeywell  Level  6  Model  43 
minicomputer  with  1 28K  of  memory,  a  10  Mbyte  disk, 
ASR-43  teletype,  2  Mast  projectors  and  various 
panel  components. 


A  key  concept  used  during  the  E-3A  MPS 
development  was  an  automated  data  base  defining 
the  possible  states  of  the  panel  components. 

The  data  base  listed  all  inputs  and  outputs  on 
each  panel,  type  of  component,  word  and  bit 
of  each  item  as  read/written  to  the  panels.  The 
data  base  was  used  to  provide  information  to 
the  functional  groups  interfacing  with  the  train¬ 
er:  software  and  the  digital  groups.  If  a 
change  occured  in  the  hardware,  the  data  base  was 
changed  and  copies  provided  to  the  users.  This 
list  was  automated  and  was  invaluable  in  inter¬ 
facing  with  the  trainer. 


Integration 


There  are  two  phases  of  integration  for  the 
maintenance  trainer.  The  first  integration  phase 
is  the  conventional  software  with  hardware  mar¬ 
riage.  The  second  phase  of  integration  is  when 
the  courseware  is  fully  integrated  with  the  soft¬ 
ware  and  hardware. 


The  software  integration  for  the  E-3A  MPS  was 
more  complex  in  that  the  courseware  interpreter. 
Procedure  Monitor,  also  had  to  be  integrated.  The 
software  integration  plan  was  multi-staged  such 
that  both  the  application  software,  and  the  Pro¬ 
cedure  Monitor  was  integrated  with  each  other 
and  then  with  the  hardware. 


During  the  software  integration,  care  must  be 
taken  to  define  appropriate  tests  so  that  any 
of  the  sof tware/ courseware  interfaces  can  be  ex¬ 
ercised.  Eventually,  all  such  interfaces  are 
required  to  be  tested. 

Courseware  integration  must  be  done  upon  com¬ 
pletion  of  software  integration.  It  involves 
system  functional  testing  as  well  as  verifying 
that  all  lesson  units  are  operational.  All 
functional  level  testing  should  be  against  re¬ 
quirements.  Lesson  units  should  be  verified 
through  operation  of  T.O.s,  lesson  plans  or  ma¬ 
terial  of  that  nature.  Annotated  T.O.s  are 
extremely  useful  to  verify  the  courseware.  Dur¬ 
ing  the  E-3A  MPS  courseware  integration,  we 
mistakingly  used  listings  to  verify  the  course¬ 
ware.  However,  verifying  the  listing  does  not 
mean  that  the  courseware  is  correct.  We  eventu¬ 
ally  used  the  T.O.  and  the  instructors  guide  to 
verify  the  lesson  units. 


During  system  integration  of  the  E-3A  MPS  it 
became  painfully  clear  that  courseware  tools 
were  required  for  debugging.  We  produce d  a  course¬ 
ware  debugger,  which  matured  during  the  E-3A  MPS 
integration  phase. 

In  the  future,  it  is  suggested  that  courseware 
tools  be  set  up  such  that  courseware  testing  can 
be  automated  to  as  large  a  degree  as  possible. 
Variability  in  student  response  leads  to  an  infi¬ 
nity  of  parameters  to  be  tested.  With  at  least 
a  minimal  amount  of  automation,  high  confidence  in 
testing  can  be  achieved. 

This  is  exactly  how  testing  evolved  on  the 
E-3A  MPS.  We  eventually  used  a  courseware 
tool,  "virtual  student",  to  verify  that  each 
branch  of  courseware  was  valid  and  would  operate 
successfully.  We  would  then  verify  the  lesson 
unit.  If  any  branch  required  additional  testing 
we  used  another  tool  to  start  at  the  correct 
courseware  step.  In  this  manner,  with  these  tools, 
we  were  able  to  complete  verification  of  all 
lessons  and  hence,  finish  system  integration. 

Conclusions 

The  conclusions  to  be  drawn  about  maintainance 
trainer  development  from  the  E-3A  MPS  experience 
are  presented  as  follows: 

1.  Provide  robust  software  architecture 
able  to  accomodate  changes  as 

well  as  additional  real-time  models. 

2.  Control  T.O.  annotation  or  lesson 
unit  generation  carefully. 

Automate  the  process,  to  at  least 
allow  for  interfacing  purposes 
and  consistency  in  documentation. 

3.  Control  PIDs/Proposal  to  system 
definition  and  then  through  require¬ 
ments  documents.  If  the  specifi¬ 
cations  are  controlled  and  communi¬ 
cated,  the  project  will  procede  more 
smoothly. 

4.  Test  to  the  PIDS  and  the  T.O.s.  Do 
not  test  to  anything  artificial 
unless  agreed  upon  with  the  customer. 

5.  To  the  largest  extent  possible, 
automate  testing.  Find  a  point 
which  provides  cost  effective 
testing  with  a  reasonable  confidence 
1 evel . 

The  E-3A  MPS  has  been  delivered  to  Keesler  AFB. 
Three  classes  have  been  given,  as  of  July  31, 
with  successful  results.  The  device  has  been 
operating  without  significant  maintenance 
problems  for  greater  than  5000  hours.  All  soft¬ 
ware  and  courseware  deficiencies  have  been 
corrected  under  the  warrenty  provision  of  the 
contract.  We  have  remained  in  close  contact 
with  the  users  and  feel  that  the  E-3A  MPS  repre¬ 
sents  a  successful  culmination  of  our  joint 
efforts. 
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In  maintenance  training.  Further  details 
may  be  found  In  Hrltz,  R.  0.,  Purlfoy, 

G.  R.,  Jr.,  and  Smith,  J.  A.  Maintenance 
Training  Simulator  Design  and  Acquisition: 
final  report.  AFHRL-TR-80-Z3.  Lowry  AFB, 
COl  Technical  Training  Division,  Air 
Force  Human  Resources  Laboratory,  April 
1980. 

OBJECTIVES 

The  Increasing  complexity  of  modern 
Air  Force  weapon  systems  and  a  decreas¬ 
ing  Air  orce  training  budget  have  com- 
blneo  produce  a  maintenance  training 
problem  that  demands  cost-effective 
Improvement  in  maintenance  trainer  design 
and  acquisition.  The  project  was  design¬ 
ed  to  address  four  objectives: 

1.  To  document  the  existing  Instruc¬ 
tional  System  Development  (ISO) 
Process,  particularly  those  pro¬ 
cedures  directed  toward  design¬ 
ing  maintenance  training  equip¬ 
ment  and  documenting  training 
equipment  requirements. 

2.  To  document  the  existing  System 
Program  Office  (SPO)  procedures 
for  acquiring  maintenance  train¬ 
ers. 

3.  To  develop  new  training  technol¬ 
ogy  and  tools  to  assist  the  ISO 
analysts  to  identify  training 
equipment  requirements  (e.g., 
the  level  of  fidelity)  and  to 
develop  a  procedure  or  mecha¬ 
nism  to  communicate  these  to 
SPO  personnel . 

4.  To  develop  procedures  to  assist 
the  Acquisition  Manager  and  his 
support  personnel  prepare  a 
procurement  specification 
suitable  for  distribution  to 
vendors  and  contractors. 

Two  of  the  objectives  were  directed 
toward  the  ISO  side  of  the  acquisition 
process,  while  the  remaining  two  objec¬ 
tives  were  directed  toward  the  SPO  side 
of  the  acquisition  process. 

APPROACH 

The  two  sides  of  the  acquisition  pro¬ 
cess  were  approached  in  a  similar  fashion. 
ISO  and  SPO  personnel  were  interviewed  to 
determine  the  existing  procedures.  Be¬ 
cause  procedures  varied  between  Air  Force 
organizations  performing  these  functions, 
model  processes  were  constructed. 

The  ISO  and  SPO  procedures  were  then 


carefully  analyzed  to  determine  problem 
areas;  areas  where  improvements  could  be 
real Ized. 

Procedures  were  developed  to  assist 
the  ISO  analyst  In  making  critical  train¬ 
ing  equipment  design  decisions  (e.g.,  de¬ 
termining  if  a  maintenance  simulator  Is 
required,  determining  the  degree  of  fidel¬ 
ity  of  the  components  to  be  represented  on 
the  trainer,  and  selecting  and  defining 
the  Instructional  features  that  are  pro¬ 
cessor-controlled  and  facilitate  managing 
the  training  situation).  All  such  proce¬ 
dures  are  presented  In  a  flow  chart  deci¬ 
sion-making  format. 

To  assist  the  ISD  analyst  in  commun¬ 
icating  the  results  of  the  ISD  analysis, 
a  model  or  generic  ISO-derived  training 
equipment  design  specification  was  devel¬ 
oped.  This  model  specification  describes 
such  training  requirements  as  the  train¬ 
ing  objectives,  a  training  application 
plan  or  model,  a  physical  and  functional 
description  of  the  components  to  be  rep¬ 
resented,  a  comprehensive  description  of 
the  processor-controlled  instructional 
features  and  a  configuration  plan  for  the 
intended  trainer. 

To  assist  the  preparer  of  the  procure¬ 
ment  specification,  a  model  or  generic 
Prime  Development  Specification  was  devel¬ 
oped.  This  generic  specification  contains 
both  engineering  and  training  requirements. 
The  training  requirements  are  derived  from 
the  ISD-derived  training  equipment  design 
specification.  The  engineering  require¬ 
ments  are  established  from  Military  Stan¬ 
dards  and  Specifications.  Accompanying  the 
generic  specification  Is  an  Appendix-Hand¬ 
book  which  provides  guidance  and  Instruc¬ 
tion  to  a  specific  application. 

Both  the  ISD-derived  and  SPO  model  spec¬ 
ification  have  the  same  format.  Since  both 
specifications  are  generic,  they  contain 
paragraphs  and  sub-paragraphs  which  are  ap¬ 
propriate  In  a  variety  of  situations  (e.g., 
to  describe  requirements  for  both  0-  and  I- 
Level  maintenance  trainers).  So  that  the 
specifications  can  be  tailored  to  any  situ¬ 
ation,  the  paragraphs  contain  blanks  to  be 
completed  by  the  preparer.  The  blanks  pro¬ 
vide  an  opportunity  for  the  preparer  to 
Insert  the  necessary  requirements  for  their 
particular  situation.  It  should  be  noted 
that  the  format  of  the  specifications  per¬ 
mit  requirements  established  In  the  ISD- 
derived  model  specification  to  be  traced 
thru  the  procurement  documentation  to  the 
final  acceptance  test  of  the  trainer.  This 
should  assure  that  the  trainer  obtained 
does,  In  fact,  provide  the  required  train¬ 
ing. 

Since  the  specifications  are  generic, 
a  set  of  Instructions  accompany  them. 
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The  Instructions  provide  guidance  on  se¬ 
lecting  the  appropriate  paragraphs  and 
sub-paragraphs  for  specific  applications. 
Also  provided  In  the  Instructions  are 
directions  for  establishing  the  needed 
requirements  (l.e.,  completing  the  blanks) 
and  Included  are  references  to  the  appro¬ 
priate  Military  Standards.  Included  In 
the  Instructions  Is  a  section  on  Lessons 
Learned— this  section  discusses  what  has 
been  learned  about  establishing  and  stat¬ 
ing  requirements  from  previous  trainer 
acquisitions. 

RESULTS 

The  ISD-project-developed  materials 
were  reviewed  by  the  3306th  Test  and 
Evaluation  Squadron  (Edwards  Air  Force 
Base,  California).  The  SPO  Project-devel¬ 
oped  materials  were  reviewed  by  personnel 
from  ASD/EN  (Wrlght-Patterson  Air  Force 


Base,  Ohio).  Comments  about  the  material 
were  generally  favorable.  The  concept  of 
having  two  specifications  was  viewed  as 
workable  and  desirable  and  as  a  way  to 
assure  that  the  ISD-derlved  training  re¬ 
quirements  would  not  be  misunderstood  or 
distorted  in  the  final  procurement 
specification.  Furthermore,  the  format 
of  the  specifications  was  very  well  re¬ 
ceived;  i.e.,  it  provided  flexibility  yet 
standardization. 

In  addition  to  describing  the  project 
activities  and  products  produced  by  the 
project,  the  Final  Report  also  discusses 
several  problem  areas  (e.g.,  the  :mpect  on 
the  ISD  analysis  from  an  accelerate  main¬ 
tenance  training  acquisition  cycle).  Nine 
problems  are  discussed.  Accompanying  the 
description  of  the  problem  is  a  list  of 
recommendations.  Areas  for  future  re¬ 
search  are  also  Identified  and  discussed. 
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THE  SIMULATOR  DATA  TEST  INSTRUMENTATION  SYSTEM 


.A  NEW  CONCEPT  IN  TRAINING  DEVICE  FIDELITY  MEASUREMENT 

William  L.  Curtice,  III 
Aeronautical  Systems  Division 
Wright-Patterson  Air  Force  Base,  Ohio 


ABSTRACT 

The  Air  Force  recently  completed  in-house  development  of  an  instrumentation  system 
designed  to  measure  flight  simulator  cue  correlation,  aero  model  fidelity,  and  dynamic 
flight  handling  characteristics.  The  Simulator  Data  Test  Instrumentation  System  (SDTIS) 
represents  a  significant  advance  in  the  state  of  the  art  for  flight  simulation  instru¬ 
mentation.  The  SDTIS  hardware  is  configured  as  a  field  transportable  "mini-system"  using 
composite  video  recording  techniques  to  concurrently  record  over  180  channels  of  analog 
or  digital  signal  data,  voice  audio,  and  television  camera  acquired  video  information. 

It  provides  immediate,  in-field,  automated  data  reduction  and  analysis.  It  permits  data 
acquired  during  simulator  test  to  be  directly  compared  to  flight  test  data  as  a  measure 
of  simulation  fidelity.  The  Air  Force  intends  to  use  SDTIS  techniques  as  an  integral 
part  of  future  simulator  test  and  specification  procedures.  This  paper  will  address  the 
SDTIS  capabilities,  functions,  and  operation  in  conjunction  with  its  application  to  future 
training  equipment  procurements. 

BACKGROUND 


Military  and  commercial  use  of  flight  simula¬ 
tors  for  crew  training  has  expanded  rapidly  in 
recent  years  in  response  to  the  rising  cost  of 
energy  and  new  aircraft.  Simulator  based  training 
systems  now  enable  commercial  airlines  to  transi¬ 
tion  crews  from  one  aircraft  to  another  (i.e., 

DC-9  to  DC-10)  with  as  little  as  one  hour  of 
actual  aircraft  flight  time.  Current  trends  indi¬ 
cate  that  non-revenue  (training)  flights  may  be 
completely  eliminated  by  some  airlines  in  the  near 
future  as  the  effectiveness  of  both  simulator 
hardware  and  related  instructional  systems  is 
improved.  Military  flight  simulation  has  tradi¬ 
tionally  been  more  demanding  of  technology  than 
its  commercial  counterpart.  When  compared  to  air¬ 
line  flight  operations,  military  flight  tasks  are 
more  difficult,  involve  more  diverse  and  complex 
systems,  and  demand  development  of  vastly  more 
complex  simulators.  Future  military  flight  crew 
training  programs  will  depend  on  use  of  a  "family" 
of  training  equipment,  ranging  from  table-top  part 
task  trainers  to  fully  integrated  weapons  systems 
trainers  (WST).  The  most  complex  of  these  simu¬ 
late  not  only  the  airframe  flight  dynamics,  basic 
flight  instruments  and  controls,  but  also  closely 
duplicate  the  control  stick  "feel",  cockpit  motion, 
vibration,  buffet,  and  the  pilot's  complete  exter¬ 
nal  visual  field  of  view.  Visual  displays  provide 
a  complete  360°  out-the-window  view  of  the  sky, 
horizon,  ground  terrain,  airbases,  ground  targets, 
ground  threats,  airborne  surface  to  air  missiles, 
and  other  aircraft.  Radar  and  electronic  viewing 
system  sensor  imagery  (forward  looking  infrared 
television  -  FLIR)  is  also  provided,  In  conjunc¬ 
tion  with  simulations  of  imagery  generated  by 
precision  guided  munitions  and  other  weapons 
release  and  guidance  systems.  Navigation  systems 
and  electronic  warfare  equipment  functions  are 
also  simulated.  Thus,  the  WST  represents  a  fully 
Integrated  trainer. 

As  pressure  to  trade-off  flying  hours  for 
simulator  time  has  intensified,  military  simulator 


users  in  turn  have  demanded  simulation  fidelity 
far  beyond  that  which  would  have  been  acceptable 
in  years  past.  This  leads  to  consideration  of 
simulator  "cue"  fidelity  and  "cue"  correlation, 
which  are  discussed  in  the  paragraphs  which  follow. 
Flight  simulators  may  be  viewed  as  closed  "man  in 
the  loop"  systems,  wherein  the  simulator  hardware 
is  expected  to  provide  accurate  cues  in  response 
to  pilots'  control  inputs.  This  concept  has 
conventionally  been  applied  to  aeronautical  per¬ 
formance  parameters;  however,  it  may  likewise  be 
applied  to  simulation  of  systems  for  navigation, 
electronic  warfare,  fire  control,  precision  guided 
munitions  control,  and  more.  The  concept  in  each 
case  is  the  same;  the  simulator  pilot  makes  inputs 
through  his  controls,  and  expects  to  perceive 
appropriate  "cues"  in  response. 

Proper  "correlation"  of  the  simulator  cues  is 
essential  for  any  simulation.  Human  physiology 
allows  us  to  sense  an  event  time  difference  as 
small  as  100  milliseconds.  For  example,  if  a  man 
operates  a  switch  controlling  a  lamp,  and  the  lamp 
does  not  illuminate  within  100  ms  after  the  man 
moves  the  switch,  he  will  be  aware  of  the  time 
delay.  Thus,  it  is  Important  that  the  simulator 
produce  accurate  cues  which  are  properly  time 
correlated  with  the  control  movements  and  with  the 
other  cues  produced.  Visual  or  motion  simulation 
systems  which  respond  to  control  Inputs  400  or  500 
milliseconds  after  the  panel  Instruments  react 
will  be  of  little  value.  Moreover,  it  is  Important 
that  the  cues  not  only  start  at  the  proper  time, 
but  also  that  their  magnitude  versus  time  be 
matched  to  the  aircraft  performance.  Matching  of 
simulator  cue  time  histories  with  those  perceived 
by  pilots  in  the  aircraft  thus  becomes  a  major 
simulator  design  task.  Failure  to  do  so  inevitably 
results  in  unhappy  simulator  pilots  who  report 
that  the  simulator  "doesn't  fly  like  the  airplane.” 
Quite  often  they  can't  tell  you  wh£  it  doesn't  fly 
right;  they  merely  perceive  a  difference.  In  most 
cases,  that  difference  can  be  quantitatively 


346 


traced  to  poor  cue  fidelity,  or  poor  cue  correla¬ 
tion.  Of  course,  simulation  is  a  cheating  game; 
it  is  not  technically  possible  to  perfectly  dupli¬ 
cate  many  cues.  Designers  must  therefore  be  both 
clever  and  innovative  in  "tailoring"  the  cues 
available  within  the  state-of-the-art  such  that 
perception  of  cue  errors  is  minimized. 

Development  of  flight  simulator  cue  fidelity 
is  largely  a  “black  art."  Each  contractor  pursues 
the  matter  a  little  differently.  Initial  success 
is  highly  dependent  on  the  quality  of  aerodynamic 
performance  data  available  from  aircraft  flight 
test  sources.  More  often  than  not,  data  available 
falls  far  short  of  that  needed  for  development  of 
a  high  fidelity  simulation.  Reasons  are  numerous. 
Quality  data  may  be  unavailable  due  to  the  fact 
that  a  simulator  is  being  developed  concurrently 
with  the  aircraft,  forcing  use  of  generic  aero 
models,  wind  tunnel  data,  iron  bird  characteris¬ 
tics,  etc.  Quite  often,  the  only  aero  data  avail¬ 
able  is  based  on  tests  of  previous  aircraft 
configurations  which  differ  considerably  from  that 
to  be  simulated.  As  a  rule,  the  only  aircraft 
test  data  recorded  and  preserved  is  that  required 
for  airframe  acceptance  or  performance  evaluation; 
data  characterizing  the  full  spectrum  of  systems 
and  cues  perceived  by  the  pilot  is  not  acquired  or 
preserved.  At  best,  data  available  allows  simu¬ 
lator  designers  to  develop  a  good  aerodynamic 
model.  However,  one  must  recognize  that  the  aero 
model  represents  only  one  link  in  the  chain  of 
signal  processing  which  occurs  between  the  pilot's 
control  input  and  cues  produced.  Flight  test  data 
is  normally  not  acquired  which  can  be  used  to 
support  the  "end  to  end"  test  concept  discussed 
above . 

Past  simulator  acceptance  tests  have  been  based 
on  performance  of  a  "family"  of  tests.  Wherever 
possible,  peripheral  systems  (such  as  electronic 
warfare  hardware)  are  tested  independently.  Limited 
tests  of  the  computer's  aero  module  (which  look 
only  at  the  aero  model,  and  do  not  consider  data 
processing  of  control  inputs,  cue  drive  outputs, 
linkage  delays,  etc.  which  occur  before  and  after 
the  aero  model)  are  performed  using  spare  computer 
digital  to  analog  outputs  and  strip  chart  recorders. 
Handling  qualities  (including  control  stick  respcnsg 
are  measured  statically.  No  multiple  axis, 
dynamic  tests  are  performed  quantitatively.  These 
tests  are  left  to  test  pilots  who  evaluate  the 
simulator  performance  subjectively.  Thus  a  pilot 
is  the  first  evaluator  of  integrated  simulator 
performance.  Government  test  procedures  have 
normally  required  a  subjective  simulator  evaluation 
by  teams  of  experienced  pilots  wherein  the  contrac¬ 
tor  was  required  to  make  hardware  and  software 
adjustments  until  the  dynamic  simulator  performance 
matches  the  pilots'  memory  of  aircraft  performance. 
Unfortunately,  pilots  became  "simulator  acclimated" 
In  just  a  few  days.  All  too  often,  simulator 
performance  acclaimed  as  adequate  by  one  team  was 
judged  inadequate  by  the  team  that  followed.  This 
subjective  procedure  usually  resulted  In  an  end¬ 
less  iteration  of  tests.  There  was  no  method  of 
tracking  a  performance  fidelity  baseline.  Such 
handling  qualities  tests  ultimately  delayed  simu¬ 
lator  deliveries  at  great  expense  to  both  contrac¬ 
tor  and  government. 

FUNCTIONAL  CONCEPT 

Functional  concepts  for  the  Simulator  Data 


Test  Instrumentation  System  were  developed  by  Air 
Force  engineers  based  on  their  experiences  in  the 
subjective  domain  discussed  above.  The  SDTIS 
embodied  two  basic  concepts.  They  were: 

1.  That  it  should  be  possible  to  quantify 
simulator  handling  qualities  and  track  handling 
qualities  changes  through  use  of  "end  to  end"  sys¬ 
tem  measurements ;  i.e.,  a  comparison  of  cue  time 
histories  versus  control  input  time  histories,  and 

2.  That,  if  given  identical  control  inputs, 
the  simulator  and  aircraft  would  produce 
"comparable"  cue  time  histories.  The  first  concept 
is  now  well  proven;  the  second  is  still  on  trial. 

Functional  SDTIS  requirements  included  the 
ability  to: 

a.  Quantitatively  measure  and  record  the 
magnitude,  phase,  and  time  relationship  of  simula¬ 
tor  cues  produced  in  response  to  standard  dynamic  , 
multi-axis  control  inputs.  The  recording  capability 
must  include  simultaneous,  synchronous  acquisition 
of  analog  and  digital  signal  data,  optically 
acquired  data  such  as  dial  pointer  positions  and 
visual  display  positions,  g-seat  and  g-suit  forces, 
motion  platform  position  and  acceleration,  stick 
position,  and  stick  force. 

b.  Make  measurements  of  cue  time  histories 
produced  in  response  to  multi -axis  control  inputs 
produced  by  a  mechanized,  automated,  simulator 
stick  (and  rudder)  mover.  The  control  movers  must 
be  programmable  to  produce  standard  reference 
inputs  (step,  ramp,  and  sine)  as  well  as  to  dupli¬ 
cate  aircraft  control  movements  made  by  test  pilots 
flying  missions  from  which  simulator  reference  data 
was  taken. 

C.  Automatically  reduce  data  recorded 
into  forms  readily  useable  by  simulator  engineers. 

In  field  co-plots  of  time  histories  were  required 
as  a  minimum.  (Cross  plots  were  subsequently  added 
as  a  requirement.) 

d.  Automatically  compare  (through  on-site, 
in-field  plots)  data  recorded  against  that  obtained 
from  flight  test  and  other  data  sources. 

e.  Self  calibrate,  automatically  scale, 
and  label  all  data  recorded. 

f.  Be  field  portable  such  that  the  system 
could  be  easily  transported  (as  airline  luggage  if 
necessary)  to  contractor  facilities  or  Air  Force 
field  sites . 

DEVELOPMENT  STATUS 

Development  of  the  SDTIS  was  initiated  in 
January  1978  as  an  in-house  activity  performed  by 
the  Visual  and  Electro-Optical  Branch  of  the 
Simulator  (Engineering)  Division  of  the  Aeronautical 
Systems  Division  (ASD).  Program  funds  and  support 
contract  management  were  provided  by  the  ASD 
Simulator  System  Program  Office.  This  effort  is 
now  nearly  complete.  Ninety  per  cent  of  the  design, 
and  50  per  cent  of  the  fabrication  was  performed 
as  an  "additional  duty"  responsibility  by  Air  Force 
simulator  engineers,  technicians,  and  cooperative 
engineering  students  who  were  also  required  to 
support  simulator  acquisition  programs.  The 
balance  was  performed  as  sub  tasks  by  a  local 
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support  contractor.  All  Initial  design  objectives 
have  been  achieved;  most  have  been  expanded. 
Development  cost  to  date  Is  approximately  $300K  for 
parts  and  contractor  support. 

SYSTEM  DESCRIPTION 

The  SDTIS  system  (less  control  movers)  Is 
shown  In  Figure  1,  as  represented  by  the  block 
diagram  of  Figure  2.  It  consists  of  a  family  of 
sensors,  video  encoding  and  decoding  electronics, 
a  video  recorder,  a  mini-computer,  graphics  termi¬ 
nal  with  hard  copy  unit,  and  a  family  of  control 
movers.  The  functions  of  each  subsystem  will  be 
discussed  below. 

Sensors 

Six  basic  sensors  have  been  provided  for  simu¬ 
lator  Instrumentation.  These  are: 

Analog  Signal  Inputs  -  The  SDTIS  system  has 
the  capacity  to  record  176  channels  of  analog  sig¬ 
nal  data.  The  signal  encoding,  cabling,  and 
filtering  subsystems  are  physically  and  function¬ 
ally  organized  into  blocks  of  sixteen  channels. 

Analog  signals  (from  simulator  electronics,  back¬ 
planes,  etc.)  are  acquired  through  the  use  of 
alligator  clips,  J-clips,  push  on  connectors,  etc., 
and  are  connected  to  terminal  blocks  on  J-boxes 
(or  spider  boxes).  See  Figure  3.  Each  J-box 
accommodates  sixteen  signals.  Fifty  foot  lengths 
of  multi-wire  cable  are  then  used  to  connect  the 
J-boxes  to  the  encoder  electronics  rack.  Analog 
channel  inputs  are  differential  with  input  imped¬ 
ances  which  exceed  one  meg  ohm  (limited  only  by  the 
cable  capacitance).  Portable,  hand  held,  battery 
operated  isolation  electronics  are  used  where 
signal  loading  is  a  problem  and  additional  isola¬ 
tion  is  required.  Simulator  signals  thus  acquired 
include  control  force  and  position  follow-up 
signals,  conmand  and  follow-up  signals  for  visual 
system  servos,  motion  platform  servos,  panel 
instruments,  and  g-seat  bellows. 

Digital  Signal  Inputs  -  The  SDTIS  system,  as 
currently  configured  has  the  capacity  to  record 
sixteen  16-bit  digital  channels.  Signal  input 
mechanics  are  configured  much  like  the  analog 
Inputs  discussed  above,  except  that  the  J-boxes 
house  active  electronics  which: 

a.  provide  additional  isolation 

b.  provide  for  digital  word  latching  triggered 
by  an  external  signal  (master  clock  or  signal 
valid),  and 

c.  accommodate  digital  inputs  of  varying 
thresholds,  polarities,  etc. 

Each  J-box  handles  one  16-bit  input. 

Motion  Sensors  -  A  motion  sense  package  has 
been  provided  for  motion  platform  acceleration 
measurements.  The  package  consists  of  a  three 
axis  rate  gyro,  three  linear  accelerometers,  and 
the  associated  electronics.  (Rate  gyros  were 
chosen  in  lieu  of  angular  accelerometers  based  on 
cost,  availability  and  maintainability.)  This 
package  would  normally  be  located  at  the  pilot's 
seat;  however,  math  exists  and  field  operable 
software  is  under  development  that  will  permit 
comparison  of  measurements  taken  from  anywhere  on 
the  motion  platform  with  flight  test  measurements 
taken  from  anywhere  on  the  airframe. 

G-sui  t  Ai r  Pressure  Sensor  -  A  g-suit  air 
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pressure  sensor  has  been  provided  to  enable  cue 
correlation  between  the  g-sult,  g-seat  signal 
drives,  and  the  motion  platform. 

External  Position  Follow-Up  Sensors  -  Retract¬ 
able  string  connected  position  sensors  have  been 
provided  for  measurement  of  control  and  servo  move¬ 
ments  in  areas  where  internal  simulator  follow-up 
signals  are  unavailable  or  cannot  be  trusted. 

Control  Force  Sensors  -  Force  sensors  have 
been  acquired  for  stick,  wheel/column  and  rudder 
control  inputs. 

TV  Cameras  -  Three  television  cameras  (one  low 
light  level  intensified  silicon  vidicon,  one 
Newvicon,  and  one  silicon  vidicon)  have  been  provi¬ 
ded  for  acquiring  cues  which  are  not  available  as 
electronic  signals.  Instrument  panel  dial  pointer 
movements  and  visual  display  movements  are  of 
primary  interest.  Many  simulator  flight  instru¬ 
ments  are  dc-servo  replications  of  actual  aircraft 
instruments  -  hence  Instrument  response  delays 
required  to  match  aircraft  instrument  performance 
(cue  correlation)  are  a  simulator  software  function. 
Simple  measurement  of  instrument  command  signals 
may  be  inadequate  for  cue  correlation  w^rk  using 
the  "end  to  end"  test  concept.  A  video  tracker, 
capable  of  either  edge  or  centroid  tracking,  has 
been  provided  to  enable  dial  pointer/visual  hori¬ 
zon  position  tracking.  The  tracker  functions  in 
conjunction  with  electronics  which  “mask"  instru¬ 
ment  or  display  geometry  which  may  confuse  the 
tracker.  For  example,  a  donut  shaped  mask  is  used 
to  track  dial  pointers,  so  that  the  tracker  is 
permitted  to  see  only  video  from  the  arc  tran¬ 
scribed  by  the  dial  pointer  tip. 

Data  Encoder/Decoder 

The  encoder/ decoder  shown  in  Figure  4  is 
responsible  for  formatting  all  of  the  sensor  sig¬ 
nals  discussed  above  into  a  composite  video  signal 
Suitable  for  recording  by  a  standard  video  recorder. 
The  composite  television  picture  format  recorded 
is  illustrated  in  Figure  5.  The  first  15  per  cent 
of  each  TV  line  is  committed  to  hosting  a  16-bit 
digital  word.  Thus,  a  "digital  block"  appears 
along  the  left  edge  of  the  TV  picture  which  accom¬ 
modates  240  words  per  TV  field.  All  signal  data 
is  recorded  in  a  digital  format  within  this 'digital 
block.”  The  remaining  TV  picture  area  (amounting 


Figure  3 

Analog  and  Digital  Sense  Boxes 


to  85  per  cent  of  the  active  TV  picture)  Is  avail¬ 
able  for  recording  of  TV  camera  video  and  for 
display  of  labeling  information.  Within  the  digi¬ 
tal  block,  176  words  are  assigned  to  analog  signal 
data,  16  words  to  digital  signal  data  and  48  words 
to  data  labeling  information.  Hardware  signal 
filtering  is  available  for  selected  analog  input 
channels  with  various  cut-off  frequencies  available 
from  30HZ  to  240HZ. 


Figure  4 

Encoder/Decoder  Electronics 
with  Built-In  Test/Set-Up  Panel 
and  Video  Tracker  Controls 


Figure  5 

SOTIS  Video  Format 

A  video  bar  graph  system  has  been  provided  as 
a  test  and  set-up  tool .  Up  to  ten  vertical  bar 
graphs  may  be  added  to  the  composite  video  picture 
Any  of  the  176  analog  channel  signals  may  be 
assigned  to  any  of  the  10  bar  graphs,  with  the  bar 
height  representing  signal  magnitude.  Bar  channel 


assignments  are  displayed  at  the  base  of  each  bar. 
The  encoder  electronics  includes  a  video  character 
generator  for  display  of  “labeling"  information. 
Labeling  is  performed  at  two  levels.  First,  each 
data  run  is  identified  with  a  label  which  appears 
continuously  in  the  video  picture.  Indicating  run 
number,  the  ever  changing  TV  field  number,  date, 
time,  and  miscellaneous  comments.  Second,  each 
signal  channel  is  labeled  with  a  64  character 
identification  used  to  describe  the  variable  name, 
source,  scaling,  etc.  Channel  labels  do  not 
continuously  appear  in  the  video  picture  as  char¬ 
acters,  although  they  are  “scrolled"  through  the 
picture  at  the  beginning  of  each  data  run.  All 
label  information  is  encoded  in  the  digital  block, 
thus  making  it  available  to  the  in-field  computer 
for  proper  labeling  of  graphs  and  hard  copy  output 
data.  All  labels,  bar  graph  assignments  and  video 
image  location  assignments  are  controlled  (entered) 
via  a  TV  typewriter  keyboard  shown  in  Figure  6. 

Two  channels  of  audio  are  concurrently  recorded 
with  the  data  discussed  above.  These  are  normally 
used  to  record  simulator  pilot  intercom  audio  and 
test  director  comments. 


Figure  6 

Television  Monitor  and 
TV  Character  Generator  Keyboard 


Video  Recorder 

The  SOTIS  design  concept  is  based  on  the  use 
of  a  commercial,  U-matlc,  3/4  Inch  cassette, 
editing  type,  video  tape  recorder  as  the  primary 
data  recording  device.  (Figure  7  -  Sony  BVU-200A) 
A  video  recorder  was  chosen  because  It: 

1.  was,  in  fact,  a  very  wide  bandwidth  data 
recorder  with  information  recording  capacities 
which  vastly  exceeded  those  required  for  this 
effort,  and 

2.  was  available  at  very  low  cost,  compared 
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to  available  Instrumentation  recorders  of  compar¬ 
able  capacity,  and 

3.  allowed  Integrated  recording  of  audio  and 
video  data  In  addition  to  the  digitally  encoded 
multi-channel  signal  data,  and 

4.  was  easily  Interfaced  to  a  digital  computer 
allowing  automation  of  data  recording  and  reduction 
tasks.  The  SDTIS  design  concept  utilizes  only  15 
per  cent  of  the  active  video  field  for  recording 
the  digital  encoded  signal  data.  This  amounts  to 

a  200  Kllloblt/second  recording  capability  of  one 
hour  duration.  If  additional  digital  recording 
capability  were  required,  the  digital  block  could 
be  expanded  over  the  total  active  picture  area  to 
achieve  a  1  megabit/second  recording  capability. 


Data  Review 

Once  recorded,  the  video  data  may  be  reviewed 
by  test  personnel  In  much  the  same  way  you  would 
review  film  with  a  film  editor.  Video  recorder 
controls  allow  bi-directional  tape  playback  at  a 
variety  of  speeds  running  from  field  by  field  stop 
motion  to  twice  real  time.  Test  directors  may 
listen  to  the  audio  and  watch  the  movement  of  dials. 
Indicators,  and  bar  graphs  representing  selected 
signal  data.  As  they  slew  through  the  tape,  they 
may  also  note  the  video  field  numbers  appearing  In 
the  run  data  block.  When  a  section  of  data  of 
particular  interest  is  located,  the  operator  then 
need  only  note  the  field  number  appearing  at  the 
start  of  the  data  of  interest. 


Figure  7 

Sony  BVU-200A  U-Matic  Video  Recorder 
Recording  Characteristics 

The  above  paragraphs  describe  a  very  flexible 
data  recording  system.  The  inherent  character  of 
the  TV  signal  format  used  provides  a  built-in  6QHZ 
signal  sample  rate  for  all  data  recorded.  In 
effect,  each  data  channel  is  digitized,  and 
recorded  sequentially  within  the  digital  block. 

This  means  that  each  channel  is  sampled,  sequen¬ 
tially  in  time  63  microseconds  apart.  The  time 
shift  of  signal  sampling  normally  provides  no 
problem  for  typical  simulator  measurements  -  as  all 
signals  are  sampled  within  16.3  milliseconds  -  a 
recording  time  accuracy  which  exceeds  all  known 
requirements  to  date.  However,  software  can  be 
provided  which  will  account  for  the  line  by  line 
signal  sample  time  offset  in  plotting.  The 
sequential  sampling  does  provide  one  advantage; 
signal  sampling  rates  greater  than  the  60HZ  TV 
fixed  rate  are  possible  if  the  same  signal  Is 
applied  to  two  or  more  channels  which  are  evenly 
spaced  within  the  TV  field.  Thus,  the  system  is 
capable  of  operating  at  a  120  or  240HZ  sample 
rate,  at  the  expense  of  the  total  number  of 
channels  available.  To  date,  ths  60HZ  rate 
exceeds  all  simulator  recording  requirements. 


Figure  8 

TI-990  Minicomputer  with  Dual  Floppy  Disk 
Tektronix  4006  Graphics  Terminal 
Tektronix  4631  Hard  Copy  Unit 


Data  Retrieval 


The  data  retrieval  system  consists  of  a  Texas 
Instrument  TI-990/10  Minicomputer  System,  a 
Tektronix  4006  Graphics  Terminal,  and  a  Tektronix 
4631  Hard  Copy  Unit.  (See  Figure  8)  The  computer 
is  a  16-bit  minicomputer  configured  with  32K  words 
of  memory  and  dual  floppy  disk  drives.  Automated 
data  retrieval  is  initiated  using  the  graphics 
terminal  keyboard  to  enter  the  beginning  field 
number  and  the  time  duration  for  the  data  of 
interest.  The  computer  will  then  take  control  of 
the  video  recorder,  search  for  the  data  of  interest, 
and  transfer  all  digital  block  signal  data  from 
the  video  tape  to  the  computer  memory.  Data  is 
then  sorted  by  channel  and  stored  on  a  floppy  disk. 
The  operator  will  then  be  prompted  to  select  the 
types  of  plots  required.  The  graphics  package 
will  co-plot  up  to  six  channels  of  time  history 
recordings,  or  will  cross-plot  (X-Y.Y.)  any  three 
channels  selected.  See  Figures  .9  and^lO  for 
examples  of  time-plot  and  cross-plot  graphics.  In 
each  case,  the  plot  packages  will  also  label  and 
scale  the  graphs  using  the  run  and  channel  labeling 
information  entered  prior  to  recording,  and  carried 
within  the  digital  block.  The  hard  copy  unit  will 
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Figure  9  Time  Plot 

reproduce  all  graphics.  It  is  significant  to  note 
that  the  plotting  package  and  retrieve  software 
process  only  sixty  data  points  (samples  across  time) 
for  each  of  the  192  channels  recorded.  The  plot¬ 
ting  resolution  of  the  graphics  hardware  cannot 
practically  handle  more  than  60  to  100  data  points. 
Thus,  if  a  data  retrieve  of  one  second  is  desired 
the  computer  will  transfer  the  plot  values  for 
each  channel  from  every  TV  field  recorded  during 
the  one  second  period  (60).  However,  if  a  two 
second  retrieve  period/plot  is  required,  the  com¬ 
puter  would  process  and  plot  data  for  all  channels 
from  every  other  field.  A  ten  second  retrieve 
would  result  in  data  being  taken  from  every  tenth 
field,  etc.  The  system  will  retrieve  and  plot  (on 
one  sheet)  data  of  any  time  Interval  from  1  second 
to  60  minutes  using  the  sixty  point  sampling 
retrieve  systems  described  above.  Plotting  of 
multiple  1  second  graphs  (most  probably  scotch 
taped  end  to  end)  is  required  in  situations  where 
every  point  recorded  must  be  plotted.  No  simulator 
applications  to  date  have  required  such  plotting 
resolution. 

Oata  Analysis 

A  very  limited  amount  of  data  analysis  capa¬ 
bility  has  been  provided  as  part  of  the  SDTIS  soft¬ 
ware.  Four  programs  will  be  available.  They  are: 

1.  Software  Filtering  -  A  three  pole  butter- 
worth  filter  Is  available  and  can  be  used  to  pro¬ 
cess  any  60  point  time  history  retrieved  from  video 
tape. 

2.  Accelerometer  Translation  -  An  accelero¬ 
meter  package  location  translation  program  Is  in 
development  which  will  allow  pilot  seat  accelera¬ 
tion  to  be  taken  from  anywhere  on  the  motion  plat¬ 
form. 

3.  Six  Post  Motion  Platform  Conversions  - 
Many  simulators  today  are  designed  with  six  post 
synergistic  motion  platforms.  These  systems  pro¬ 
vide  six  degrees  of  freedom  movement,  but  the 
axis  of  platform  motion  is  usually  the  synergistic 
product  of  the  movement  of  six  hydraulic  drives. 

All  simulator  computer  software  Is  structured  to 
drive  these  six  hydraulic  cylinders,  with  signals 
which  are  easily  recorded.  However,  review  of  the 
six  post  drive  values  tells  one  very  little  -  what 
Is  desired  are  the  resulting  values  of  platform 
pitch,  roll,  yaw,  and  heave  (x,  y,  z).  Transfor¬ 
mations  have  been  developed  for  this  conversion. 
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Figure  10  Cross  Plot 

Thus,  the  six  post  drive/follow  signals  can  be  used 
to  produce  time  histories  of  platform  motion  which 
can  be  directly  compared  to  other  simulator  flight 
performance  parameters  or  aircraft  data. 

4.  X-Y  Position  to  Angle  Conversion  - 
Routines  for  plotting  video  tracker  X-Y  position 
outputs  as  a  function  of  dial  pointer  angle. 

Extensive  data  analysis  is  possible  If  the 
SDTIS  is  linked  to  a  larger  computer  system  with  a 
modem.  This  will  permit  use  of  techniques  such  as 
Fast  Fourier  analysis  which  are  generally  beyond 
the  capability  of  the  SDTIS  minicomputer.  Software 
will  be  developed  which  will  allow  the  SDTIS  system 
to  act  as  a  data  terminal  to  the  General  Electric 
Timeshare  network.  This  arrangement  should  give 
simulator  engineers  the  best  of  both  worlds  - 
independent  data  reduction  to  the  maximum  extent 
possible,  coupled  with  the  power  of  a  large  system 
when  and  where  needed. 

Control  Movers 

Previous  sections  of  this  paper  have  described 
the  basic  functions  and  design  of  the  SDTIS  -  to 
sense  cues,  to  record  them,  and  to  conveniently 
reproduce  them  as  hard  copy  plots.  However,  the 
SDTIS  (or  any  other  instrumentation  system)  is 
Incapable  of  fulfilling  its  Intended  "end  to  end” 
test  roll  without  the  ability  to  provide  standard, 
controlled  Inputs  for  the  simulator  controls  (stick 
and  rudder).  Comparison  of  cue  time  history  data 
taken  from  run  to  run  on  the  same  simulator  Is  not 
possible  unless  the  control  Inputs  (which  caused 
the  cues  to  be  produced)  were  nearly  identical. 

Even  more  significantly,  comparison  of  simulator 
cue  time  histories  with  flight  test  data  Is  not 
possible  unless  the  simulator  is  flown  under  the 
same  Initial  conditions,  and  with  the  same  control 
Inputs,  as  the  flight  test  airplane.  Such 
controlled,  standardized  inputs  are  not  possible 
using  human  operators  (pilots).  Thus,  we  see  the 
need  for  a  family  of  control  movers  that  are  capa¬ 
ble  of: 

1.  Three  axis  dynamic  control  Inputs 

2.  Making  standard  control  Inputs  which  are 
likely  to  produce  known  results  (ramp,  sine,  etc.) 

3.  Reproducing  aircraft  control  inputs  as 
recorded  by  flight  test  Instrumentation,  and 

4.  Operating  In  either  a  position  or  force 
mode  (force  versus  time  or  position  versus  time). 
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Figure  11  shows  the  yoke  and  rudder  movers  with 
associated  controls  and  electronics  designed  for 
this  purpose.  Test  and  system  Integration  of  the 
movers  Is  now  ongoing.  When  fully  integrated,  the 
control  movers  will  be  driven  by  prerecorded  con¬ 
trol  movement  profiles  which  have  been  stored  on 
floppy  disks.  Use  of  the  movers  will  then  complete 
our  ability  to  perform  a  controlled  "end  to  end" 
simulator  system  test.  Figure  12  shows  the  yoke 
mover  coupled  to  a  control  yoke.  Note  the  force 
sensor  located  near  the  universal  joint. 


Figure  11 

Control  Hover  Systems 

(Yoke  mover.  Rudder  mover  and  associated  electronics) 


This  completes  the  description  of  the  SOTIS. 
As  illustrated  in  the  photograph,  all  instrumen¬ 
tation  hardware  has  been  packaged  such  that  it 
“could  be"  shipped  as  airline  baggage  by  most 
carriers.  However,  the  total  system  weight 
exceeds  2000  pounds.  Air  freight  and  truck  are 
expected  to  be  most  often  used. 


Figure  12 

Yoke  Hover  Coupled  to  T-40  Simulator  Yoke 


FUTURE  SYSTEMS-DATA  BUS  ARCHITECTURE 

The  current  design  concept  of  the  SDTIS  does 
little  to  consider  the  internal  system  architecture 
of  the  simulator.  Of  course,  point  to  point  signal 
tracing  is  possible  -  and  was,  in  fact,  a  design 
objective.  However,  as  simulator  systems  evolve, 
we  expect  to  see  the  current  analog  interface  sys¬ 
tems  replaced  with  architecture  wherein  both 
control  inputs  and  cue  producers  (instruments  and 
displays)  are  connected  directly  to  a  digital  data 
bus.  Thus,  the  signal  tracing  function  will 
largely  be  replaced  by  a  data  bus  monitoring  func¬ 
tion.  Such  is  already  the  case  with  much  elec¬ 
tronic  warfare  (EW)  simulation.  The  ability  to 
eavesdrop  on  data  bus  communications  is  essential 
to  system  test  and  troubleshooting.  An  extension 
of  the  SDTIS  to  encompass  data  bus  monitoring  is 
now  in  design  for  the  EW  task.  However,  the  basic 
"end  to  end"  test  concept  of  the  SDTIS  will  remain 
unchanged.  It  is  interesting  to  note  too,  that  as 
aircraft  systems  architecture  follows  the  data  bus 
trend,  high  fidelity  simulation  may  be  even  more 
difficult  to  produce.  Aircraft  controls  and  dis¬ 
play  technologists  are  now  considering  hierarchal 
data  bus  structures,  wherein  data  is  processed  (by 
asynchronous  processors)  and  transferred  from  bus 
to  processor  to  bus  several  times.  Proper  simu¬ 
lation  of  cue  correlation/timing  for  such  systems 
may  indeed  by  a  challenge. 

CONCLUSIONS 

Simulator  Procurement  Impact 

SDTIS  availability  is  expected  to  significantly 
impact  the  Air  Force  procurement  of  flight  simula¬ 
tors.  Trainer  handling  qualities  tests  are 
expected  to  transition  from  a  largely  subjective 
activity,  to  a  largely  quantitative  one,  although 
subjective  evaluation  of  handling  qualities  will 
probably  never  be  eliminated  completely.  Use  of 
SDTIS  test  procedures  will  be  written  into  future 
procurement  specifications  and  the  specifications 
themselves  will  be  written  in  more  quantitative 
terms  than  has  been  possible  in  the  past.  However, 
the  degree  of  simulation  fidelity  achievable 
through  use  of  SDTIS  concepts  is  dependent  on  the 
degree  to  which  the  flight  test  instrumentation 
conmunity  can  support  simulator  designers  with 
high  quality  data.  To  date,  the  SDTIS  has  been 
used  by  the  Simulator  SPO  for  evaluation  of  both 
the  Boeing  and  Singer  B-52  Weapons  Systems  Trainers, 
and  by  the  Air  Force  Flight  Test  Center  for  evalu¬ 
ation  of  an  F-4  Simulator  at  Moody  AFB,  Georgia. 

Flight  Test  Impact 

Use  of  SDTIS  concepts  in  simulator  procurement 
is  expected  to  expand  significantly  during  the 
1980s.  Thus  there  will  be  a  demand  for  flight  test 
data  required  to  support  the  high  fidelity  inte¬ 
grated  aircraft  system  simulation  discussed  at  the 
beginning  of  this  paper.  This  will  likely  impact 
the  flight  test  instrumentation  community  in  the 
form  of  requests  for  data  from  more  sensors 
located  in  or  near  the  cockpit,  which  are  correlated 
with  data  obtained  from  other  aircraft  systems 
(engine,  airframe,  etc.).  For  example,  accurate 
dynamic  recording  of  stick  and  rudder  position  and 
force  correlated  to  instrument  readings,  throttle 
setting  and  flight  dynamics  will  be  extremely  valu¬ 
able.  Likewise,  recordings  of  vibration,  buffet, 
and  g- forces  experienced  by  the  pilot  and 
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correlated  to  other  flight  dynamics  will  be  criti¬ 
cal  to  development  of  quality,  dynamic  motion  sim¬ 
ulation.  The  nature  of  flight  test  data  processing 
and  massage  may  also  be  impacted.  Complete  time 
histories  of  control  force  or  position  may  be 
needed  In  areas  where  only  peak  values  were 
required  before.  This  is  Illustrated  by  an  example 
of  pitch  trim  force.  During  level  flight,  when 
flaps  are  dropped,  most  aircraft  will  naturally 
assume  either  a  pitch-up  or  pitch-down  condition. 
The  pilot  must  Input  some  amount  of  control  stick 
force  to  pull  the  nose  back  to  straight  and  level. 
Aircraft  specifications  are  usually  concerned  only 
with  the  peak  stick  force  required  to  resume  level 
flight.  The  only  data  which  customarily  shows  up 
in  the  flight  test  report  is  that  number  -  a  single 
value.  However,  development  of  good  simulation 
requires  a  knowledge  of  the  complete  stick  force 
time  history  required  to  trim  the  aircraft.  Thus, 
we  can  foresee  a  need  for  not  only  more  data  (from 
new  sensors  at  additional  locations)  but  also  for 
expansions  In  reporting  data  that  Is  currently 
recorded. 

Application  of  SDT1S  to  Flight  Test 

The  current  SDTIS  hardware  was  not  designed  to 
fly  (although  it  could  probably  be  tested  and 
certified  for  airborne  use  assuming  an  aircraft 
had  space  to  hold  it).  However,  techniques  deve¬ 
loped  for  the  SDTIS  may  provide  solutions  to  some 
problems  of  concern  within  the  flight  test  commun¬ 
ity.  Of  particular  interest  should  be  the  use  of 
digital  encoded  video  recording  techniques. 

However,  the  SDTIS  represents  only  one  approach  to 
exploiting  a  video  technology  that  promises  much 
capability  at  relatively  low  cost.  Development 


of  digital  video  sensors  and  recorders  is  now 
moving  quickly  due  to  the  support  of  consumer  and 
broadcast  marketing  Interests.  This  work  promises 
to  yield  future  generations  of  advanced  recording 
devices  which  may  significantly  surpass  the  capa¬ 
bilities  of  lesser  developed  recording  technologies 
used  conventionally  for  flight  test  Instrumentation, 
The  video  techniques  developed  for  the  SDTIS  should 
warrant  serious  consideration  by  designers  of 
future  flight  test  Instrumentation  systems. 

SUMMARY 

Development  of  the  Simulator  Data  Test 
Instrumentation  System  is  nearly  complete.  The 
instrumentation  hardware  and  the  test  methodology 
it  supports  represent  an  extremely  flexible  tool 
which  may  assist  both  government  and  Industry  in 
development  of  training  systems  which  exhibit  the 
performance  fidelity  required  in  the  1980s. 
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ABSTRACT 


Problems  involved  in  the  design  of  flight  simulator  displays  have  received  insufficient 
attention.  Simulator  utilization  and  training  problems  have  a  basis  in  the  design  process. 
It  is  generally  left  to  the  simulator  designer  to  use  'best  judgement'  in  translating  train¬ 
ing  requirements  to  display  software/hardware  specifications.  At  present,  human  factors 
data  needed  to  accomplish  this  are  either  not  available  or  simply  not  accessible  in  a  form 
useful  to  the  designer.  Basic  sensory  and  perceptual  data,  principles,  and  models  exist 
or  can  be  derived  which  could  serve  as  a  valuable  resource  for  the  designer  in  the  specifi¬ 
cation  of  display  requirements.  The  Integrated  Cuing  Requirements  Study  is  a  current  Air 
Force  sponsored  effort  to  consolidate  and  publish  these  data  in  a  format  which  will  enable 
designers  to  rapidly  access  and  apply  this  information. 


INTRODUCTION 

"Psychologists  should  be  able  to  say  what 
needs  to  be  simulated  and  engineers  should  be  able 
to  say  what  can  be  simulated." 

"In  the  absence  of  guidance  from  psycholo¬ 
gists,  engineers  can  hardly  do  more  than  strive 
for  realism  limited  by  technical  consideration 
and  cost."  (1) 

In  the  past  few  years,  the  importance  of 
simulation  to  aircrew  training  has  greatly  accel¬ 
erated  as  have  the  engineering  state  of  the  art 
and  relative  procurement  costs.  However,  rela¬ 
tively  little  advancement  has  been  made  in  provid¬ 
ing  an  objective  basis  for  translating  training 
requirements  Into  aircrew  training  device  specifi¬ 
cations.  Much  of  the  process  as  it  currently 
exists  is  dependent  upon  the  subjective  judgement 
of  experienced  simulator  engineers  in  the  absence 
of  sufficient  behavioral  ( i . e. ,  sensory  and 
perceptual)  resource  data.  In  fact,  the  driving 
considerations  are  frequently  the  most  advanced 
equipment  features  that  can  readily  be  produced. 

The  Air  Force  Human  Resources  Laboratory, 
Aerospace  Medical  Research  Laboratory,  and 
Aeronautical  Systems  Division's  Deputy  for  Simula¬ 
tors  and  Engineering  Directorate  have  jointly 
sponsored  a  study  directed  at  the  exploitation  of 
sensory  and  perceptual  characteristics  In  the 
design  of  visual  displays  for  aircrew  training 
devices.  Based  upon  this  research.  It  Is  clear 
that  sensory/perceptual  data,  principles  and 
models  do  exist  (or  can  be  derived)  which  could 
assist  simulator  engineers  in  the  design  of 


cost  and  training  effective  simulators. 

DESIGN  PHILOSOPHY  AND  PROCESS 
Systems  Complexity 

The  primary  purpose  of  a  flight  simulator  is 
to  synthesize  information,  relevant  to  the 
training  and  performance  of  piloting  activities, 
that  is  normally  available  to  the  aircrew  in  the 
operational  aircraft  environment.  This  Informa¬ 
tion  is  typically  imparted  to  the  aircrew  member 
at  the  flight  station  via  a  synchronized  array  of 
visual,  motion,  and  aural  displays.  These 
displays  interface,  in  turn,  with  a  variety  of 
non-display  components. 

The  basic  components  of  a  flight  simulator 
typically  Include  a  flight  station,  visual 
display  subsystem,  acceleration  and  motion 
subsystem,  control  simulation  subsystem,  sensor 
simulation  subsystem.  Instructional/experimental 
control  subsystem,  and  computational  subsystem. 

Each  of  these  system  components  is  further 
reducible  to  subsystems,  assemblies,  subassemblies, 
etc.  (Figure  1).  The  Flight  Station  Includes 
status  lamps  and  Indicators,  radar/sensor 
displays,  and  flight  Instruments.  The  Visual 
Display  Subsystem  is  Interfaced  with  the  Flight 
Station  to  provide  a  through  the  windscreen  view 
of  the  external  flight  environment  (2).  The 
Acceleration  and  Motion  Subsystem  simulates  the 
non-visual  environmental  forces  which  act  upon 
the  aircrew  member  in  operational  aircraft. 

The  feel  of  the  controls,  mimicking  environmental 
forces  such  as  aerodynamic  loading  of  the  control 
surfaces,  is  displayed  via  the  Control  Simulation 
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or  'Control  Loading'  Subsystem. 

Interactions  occur  among  components  and/or 
levels  of  a  given  subsystem  as  well  as  between 
Individual  subsystems  (Figure  1).  These  Inter¬ 
actions  must  be  anticipated  and  specified  In  the 
design  process  prior  to  the  actual  Integration  of 
these  components  into  a  functional  simulation. 

Design  Process 

It  Is  the  critical  task  of  the  simulator 
designer  to  translate  training  (or  research) 
requirements  Into  display  system/subsystem 
specifications.  Training  requirements  are 
identified  and  reduced  by  task  and  cue  analyses 
into  simpler  behavioral  elements.  This  task/cue 
analysis  results  In  the  specification  of  aircrew 
information  requirements  which,  In  turn,  is 
expected  to  lead  to  a  specification  of  what  must 
be  displayed  (i.e.,  content),  and  the  necessary 
characteristics  of  the  display  (i.e.,  quality  and 
format).  These  must  then  be  translated,  by  the 
designer,  into  simulator  system/ subsystem 
specifications. 

The  problem  is  that  in  actual  practice 
there  is  Insufficient  information  available  to 
the  designer  to  enable  this  logical  and  orderly 
design  process  to  work.  It  is  left  up  to  the 
designer  to  use  his  'best  judgement'  in  those 
areas  where  data  are  lacking.  Typically,  design 
decisions  are  made  on  the  basis  of  a  phenomeno¬ 
logical  integration  of  a  set  of  variables  that 
are  not  necessarily  optimal  in  terms  of  satisfy¬ 
ing  training  requirements  (Figure  2).  These 
include  the  state-of-the-art  technology,  past 
approaches,  cost/performance  trade-offs, 
management  constraints,  and  human  factors 
guidelines  to  the  extent  that  they  are  available 
in  a  useful  format. 

Design  Philosophy 

As  can  be  seen  from  Figure  2,  critical 
decisions  made  in  the  design  process  are  based 
on  the  subjective  integration  of  a  range  of 
variables  by  the  system  or  subsystem  designer. 

In  the  absence  of  sufficient  hard  data,  the 
designer  must  make  some  basic  assumptions  about 
what  information  is  necessary  to  satisfy  training 
requirements,  the  approach  to  portraying  that 
information  (i.e.,  display  content)  and  the 
necessary  quality  and  format  of  the  display. 

Typically,  the  approach  to  simulator  design 
has  been  to  provide  as  much  of  a  linear  equiva¬ 
lent  of  the  real  world  (i.e.,  realism)  as  the 
state-of-the-art  technology  will  permit  (3). 

This  of  course  is  done  with  the  expectation  that 
satisfactory  compliance  with  training  require¬ 
ments  is  positively  correlated  with  apparent 
realism  (4).  In  other  words,  if  a  simulation 
looks,  feels  and  sounds  like  the  real  aircraft, 
then  the  basic  requirements  for  training  should 
likewise  be  present.  The  design  engineer  can 
hardly  be  faulted  here  since  he  gets  his  direc¬ 
tion  from  a  customer  (the  operational  user)  whose 
criteria  for  acceptability  is  based  upon  the 
degree  to  which  the  ground  based  simulated 
environment  approximates  his  perception  of  the 
Veal'  flight  environment  (5). 


Implementation  of  this  maximum  realism 
approach  has  fallen  considerably  short  of  its 
ideal  (i.e.,  an  objective  simulation  of  the  flight 
environment).  As  evidence  of  this,  the  most 
advanced  computer  generated  visual  displays  are, 
at  best,  low  fidelity  representations  of  the  real 
world.  These  typically  appear  cartoonish  and 
deficient  in  fine  detail.  In  motion  simulation, 
platforms  are  pushed  to  their  maximum  excursion 
(within  facility  constraints)  so  as  to  display  the 
aircraft  motion  environment  with  as  much  realism 
as  possible  before  commencing  washout.  Regardless 
of  these  attempts,  motion  displays  are  beset  with 
false  motion  information  which  is  inconsistent 
with  the  aircraft  motion  environment.  Addi¬ 
tionally,  there  are  numerous  instances  of  spatial/ 
temporal  asynchrony  between  the  visual  and  motion 
display  modes.  In  sum,  there  exist  many  examples 
of  what  we  call  errors  of  omission,  Inclusion, 
and  synchronization  that  detract  from  the  realism 
of  the  display. 

Errors  of  omission  generally  refer  to  those 
spatial /temporal  details  that  may  be  lacking  In  a 
display  which  detract  from  apparent  realism. 

Visual  errors  of  omission  Include  limitations  in 
texture,  density  of  scene  content,  size  and  con¬ 
tinuity  of  the  visual  field,  aerial  perspective, 
resolution  and  directional  illumination  (shadows). 
In  motion  simulation, these  include  high-G  loading 
and  sustained  motion. 

Errors  of  inclusion  are  those  artificial 
features  or  details  unique  to  the  simulation  that 
are  not  commonly  found  In  the  ‘real  world'. 

Errors  of  inclusion  for  visual  displays  Include 
saturated  colors,  level  of  detail  switching, 
visible  raster  patterns,  and  stylized  scene  com¬ 
ponents  that  tend  to  be  uniform  in  shape  or 
distribution  throughout  the  display.  Similarly, 
in  motion  displays  there  are  anomalies  related  to 
motion  washout  and  to  equipment  limitations 
(e.g.,  'hydraulic  bump'). 

Errors  of  synchronization  refer  to  spatial 
and/or  temporal  phase  lags  within  or  across 
display  modes.  These  include  delays  and/or 
conflicting  information  among  control  interface, 
visual,  and  motion  systems. 

It  is  Important  to  emphasize  that  these 
errors  are  errors  only  in  that  they  depart  from 
the  Intended  realism  of  the  display.  The  actual 
effects  that  these  have  on  performance  vary 
widely  and  are  in  need  of  further  study.  However, 
negative  effects  have  been  documented  ranging  from 
temporary  distractions  to  which  a  pilot  quickly 
adapts  (6)  to  incidents  of  severe  psychophysio- 
logical  distress  (7).  On  the  other  hand,  planned 
distortions  or  manipulations  of  the  visual  scene 
have  been  shown  to  enhance  performance  in  the 
flight  simulator  (8,9).  Several  studies  have 
reported  positive  effects  on  training  or  perfor¬ 
mance  by  systematically  augmenting  the  Information 
available  to  the  pilot  in  the  simulated  visual 
scene  In  order  to  facilitate  the  acquisition  of 
task  critical  Information  (10,11). 

Another  more  recent  approach  has  been  to 
tailor  display  specifications  to  the  information 
requirements  of  the  aircrew  member.  Presumably 
this  would  enable  the  optimal  allocation  and 
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prioritization  of  technology  constrained  display 
resources.  A  major  problem  is  that  at  present 
there  are  no  reliable  or  comprehensive  procedures 
for  determining  Information  requirements  for  the 
training  or  performance  of  specific  aircrew 
activities.  While  it  is  possible  through  task/cue 
analyses  (including  geometrical  analyses  of  the 
task  environment)  to  define  the  information 
available  to  the  aircrew  to  support  the  realism 
approach.  It  is  not  well  understood  which  informa¬ 
tion  is  In  fact  relevant  given  the  aircraft  system, 
environment,  mission,  and  a  host  of  individual 
variables  relating  to  the  aircrew  member  and  his 
previous  training.  Hence,  given  the  current  state 
of  knowledge,  it  remains  for  the  designer  to 
bridge  the  gap  between  training  requirements  and 
information  requirements.  In  the  absence  of 
sufficient  guidance  from  psychologists,  the 
designer  has  no  choice  but  to  use  reality  as  his 
metric. 

Once  the  designer  has  made  these  decisions 
regarding  display  information  requirements,  he 
must  decide  how  this  information  should  be  repre¬ 
sented  in  the  display.  Unfortunately,  there  is  no 
objective  methodology  for  generating  pictorial 
scene  requirements  from  visual  information  require¬ 
ments  (11,12).  Here,  once  again,  the  designer  is 
called  upon  to  use  his  best  judgement  based  upon  a 
subjective  integration  of  variables  (Figure  2) 
that  are  not  optimal  for  satisfying  the  aircrew 
member's  perceptual  information  requirements. 

USE  OF  SENSORY  AND  PERCEPTUAL 
DATA  AS  A  DESIGN  RESOURCE 

Once  the  designer  has  established  what  is 
required  and  how  it  should  be  portrayed,  it  still 
remains  for  him  to  specify  the  parameters  of  dis¬ 
play  quality,  format,  and  configuration  relative 
to  the  total  system.  The  issues  of  display 
quality  are  essentially  information  management 
issues.  One  approach  to  effective  display  manage¬ 
ment  of  aircrew  information  requirements  is  to 
capitalize  on  human  sensory  and  perceptual  capa¬ 
bilities  and  limitations. 

The  sensory  and  perceptual  data  are  specifi¬ 
cally  germane  to  the  needs  of  the  simulator 
engineer  or  designer.  These  data  provide  func¬ 
tional  relationships  for  the  variables  that  influ¬ 
ence  the  acquisition  and  processing  of  information 
as  well  as  perceptual  motor  control  output  (13,14). 
For  example,  in  determining  the  information 
requirements  for  an  'out  the  window’  target 
interception  task: 

a.  The  information  acquisition  or  sensory 
data  describe  the  variables  which  influence  target 
detection  (e.g.,  luminance,  contrast,  target  size). 

b.  The  information  processing  or  perceptual 
data  describe  those  variables  which  impact  identi¬ 
fication  of  the  target  (e.g.,  perception  of  size, 
distance,  motion). 

c.  The  perceptual  motor  control  data  define 
the  variables  which  influence  the  manual  control 
of  the  aircraft  while  tracking  the  target. 

When  used  appropriately,  these  data  can  be  a 
valuable  resource  for: 


1.  Developing  specifications  based  on  sen¬ 
sory  or  perceptual  characteristics  ( i . e. , 
matching  simulator  display  characteristics  to 
human  sensory  capabilities).  For  example,  it 
should  not  be  necessary  to  simulate  a  specific 
force  of  0.01  G  since  this  is,  under  most  condi¬ 
tions,  below  the  threshold  of  detectability. 

2.  Evaluating  specifications  or  prioritizing 
design  options.  Many  existing  specification 
requirements  and  industrial  standards  do  not  have 
an  empirical  basis  for  their  existence.  The 
sensory  and  perceptual  data  can  be  a  valuable 
resource  for  their  evaluation.  In  addition,  past 
attempts  to  achieve  realism  have  resulted  in 
spatial/temporal  distortions  which  could  have 
negative  impact  on  the  acquisition  or  processing 
of  cue  information.  The  sensory  and  perceptual 
data  can  provide  a  basis  for  evaluating  this 
impact.  More  importantly,  capitalizing  on  the 
information  processing  data  might  provide  a  basis 
for  alternatives  to  realism. 

3.  Generating  new  design  or  training  alter- 
natives.  Data  from  Reqan,  Beverley,  and  Cynader 

( 1 979; 15),  Regan  (1980;16),  Ginsberg  (1980;17,18) 
and  others  suggest  that  specific  sensory  capabili¬ 
ties  may  be  enhanced  through  special  training 
procedures.  This  portends  a  new  approach  to 
pilot  training  as  well  as  a  new  generation  of 
training  devices  which  are  geared  toward  improving 
the  pilot's  'natural'  ability  to  acquire  and  pro¬ 
cess  information. 

When  used  appropriately,  the  sensory  or 
perceptual  data  can  be  an  effective  resource  to 
the  experienced  engineer  and  designer.  However, 
there  are  limiting  factors  to  the  value  of  these 
data.  Specifications  suggested  by  these  data  may 
not  be  practical  in  terms  of  technology  or  cost. 

In  fact,  in  many  instances  current  technology  can¬ 
not  match  the  limitations  of  human  perception. 

As  an  example,  consider  the  situation  in  Computer 
Generated  Imagery  wherein  the  displayed  image  of  a 
light  source  is  decreased  in  area  as  the  square  of 
the  calculated  viewing  distance  so  as  to  provide  a 
change  in  retinal  image  size  that  conforms  with 
normal  visual  experience.  The  displayed  image 
cannot  be  reduced  below  one  pixel  which,  for  most 
displays,  subtends  an  angle  two  to  four  times 
larger  than  the  optimal  resolution  limit  (19). 

Appropriate  implementation  of  these  data 
requires  that  they  be  integrated  with  other 
factors  by  the  experienced  engineer  and  designer 
(as  shown  in  Figure  2)  to  yield  final  requirements 
or  specifications. 

Furthermore,  much  of  the  data  regarding  human 
sensory  limitations  and  perceptual  capability  are 
either  not  yet  known  or  have  not  been  sufficiently 
well  defined  to  enable  their  direct  translation  to 
engineering  applications.  Where  data  are  known, 
they  are  usually  not  readily  accessible  in  a  form 
useful  to  simulator  designers  (20,21,22).  It 
should  be  noted,  however,  that  where  visual  sensi¬ 
tivity  data  have  been  accessible  to  designers  (23, 
24),  they  have  been  successfully  exploited  in  the 
specification  of  visual  displays  (25). 

The  Integrated  Cuing  Requirements  Study  is  a 
Research  and  Development  program  under  contract  to 
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The  Boeing  Aerospace  Company  (Seattle  WA)  to 
exploit  sensory  and  perceptual  characteristics, 
principles  and  models  useful  to  the  design  and 
specification  of  simulator  displays.  The  techni¬ 
cal  objectives  of  this  study  are: 

1.  To  identify,  collect  and  consolidate 
basic  sensory/perceptual  data  germane  to  training 
and  engineering  requirements.  This  will  be 
documented  as  a  data  base  organized  around 
principal  areas  of  sensation  and  perception. 

It  will  contain  a  minimum  of  text  but  will 
instead  concentrate  on  detailed  illustrations, 
quantitative  functions,  characteristics  and 
models. 

2.  To  develop  specialized  information 
retrieval  techniques  to  facilitate  the  accessi¬ 
bility  and  application  of  these  data  by  design 
engineers.  These  will  be  documented  as  a  User's 
Guide  which  will  lend  structure  to  the  Data  Base 
in  accordance  with  the  needs  of  simulator 
designers.  It  is  also  feasible  to  develop  User's 
Guides  to  the  Data  Base  for  other  user  popula¬ 
tions  for  whom  these  data  could  be  a  resource 
(e.g. ,  human  factor  engineers). 
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Generated  Displays.  Abstract  from  SPIE, 

Visual  Simulation  and  Image  Realism,  Vol  162, 
8-11,  1978. 

10.  Lintern,  G. ,  Transfer  of  Landing  Skill  After 
Training  with  Supplementary  Visual  Cues. 

AFOSR  TR-78-2,  July  1978. 


11.  Semple,  C.A.;  Hennessy,  R.T.;  Sanders,  M.S.; 
Cross,  B.K. ;  and  Beith,  B.H.,  Simulator 
Fidelity  Features:  Simulator  Training 


Requirements  and  Effectiveness  Study  (STRES). 
AFHRL-TR-80-XX.  (in  Press") 


In  summary,  the  Integrated  Cuing  Require¬ 
ments  Study  will,  in  itself,  provide  a  valuable 
resource  to  design  engineers.  However,  optimal 
implementation  of  this  resource  to  training 
simulator  displays  is  wholly  dependent  on  also 
developing  the  ability  to  specify:  (1)  the 
information  necessary  for  effective  training  of 
specific  aircrew  tasks,  and  (2)  the  optimal  way 
of  portraying  this  information  ( i . e. ,  levels  of 
abstraction  or  realism).  A  research  effort  which 
systematically  responds  to  these  basic  questions 
is  critical  to  the  design  of  training  effective 
flight  simulators. 
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Figure  2.  Design  decision  process.  Design  requirements  and  specifications  are  determined 
by  the  subjective  integration  of  a  range  of  variables. 
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ABSTRACT 

Research  was  conducted  to  optimize,  in  terms  of  both  hardware  response  and  soft¬ 
ware  driving  philosophies,  the  effectiveness  of  pneumatically  operated  g-seat  and  g- 
suit  flight  simulation  subsystems  for  high  performance  aircraft,  such  as  the  Advanced 
Simulator  for  Pilot  Training  (ASPT)  F-16.  Results  indicate  that  drive  philosophies 
should  be  a  dynamic  functlor  of  g-range,  ground  or  air  operation,  maneuver,  and  task. 
In  some  cases  logical  and  satisfactory  philosophies  for  T-37  training  were  improved  by 
reversing  their  effect  for  high  g  maneuvers. 


INTRODUCTION 


The  G-seat  is  a  flight  simulation  device 
designed  to  induce  body  position  and  pressure 
manipulations,  controlled  according  to  a  pro¬ 
grammed  drive  philosophy,  so  that  acceleration 
sensations  may  be  perceived  by  pilots.  The 
typical  pneumatic  G-seat  system  consists  of  a 
seatpan  with  16  active  air  bellows,  a  backrest 
with  nine  active  bellows,  thigh  panels  with  six 
active  bellows,  and  an  active  seat  belt. (4) 

Each  of  the  32  items  is  individually  controlled 
via  pneumatic  hoses  leading  from  the  pneumatic 
control  components  to  the  seat  elements.  The  32 
pneumatic  devices  are  in  turn  governed  by  a 
G-seat  computer  program  within  the  simulator's 
computational  system. 

Background 

The  Advanced  Simulator  for  Pilot  Training 
(ASPT)  G-seat  was  developed  by  Singer-Link  to  be 
used  in  conjunction  with  a  T-37  flight  simula¬ 
tion.  As  such,  it  was  designed  for  sustained 
G-cueing  during  basic  airwork  and  aerobatic 
flight  typical  of  training  aircraft.  The  ASPT 
force-cueing  philosophy  was  to  provide  onset 
acceleration  cues  with  a  six  degree-of-freedom 
platform  motion  system  followed  by  blending  in 
the  G-seat  cues  to  sustain  the  acceleration 
effect  as  the  motion  system  cues  were  "washing 
out".  The  original  software  was  based  on  the 
geometry  and  characteristics  of  T-37  aircraft; 
however,  the  seat  was  also  designed  as  a 
research  device  with  a  high  degree  of  flexi¬ 
bility  in  order  to  investigate  the  simulation  of 
tactile,  pressure,  and  skeletal  posture  cueing 
resulting  from  flight-induced  body  G-loading. 

Purpose 

Although  advantage  could  be  taken  of  this 
‘Texibility  in  configuring  the  G-seat  responses 
to  other  aircraft  characteristics,  it  was  also 
necessary  to  improve  the  response  time  of  the 
system  to  effectively  simulate  the  G-cues  of 
higher  performing  F-16  aircraft.  In  addition  it 
was  felt  from  the  early  sensory  mechanism  model¬ 
ing  work  that  the  G-seat  could  be  an  effective 
onset  cueing  device.  Since  the  G-seat  interfaces 


with  the  most  responsive  sensory  mechanisms,  the 
tactile  and  pressure  receptors,  it  can  be  argued 
that  it  should  be  designed  as  the  most  responsive 
of  the  G-cueing  devices.  A  highly  responsive 
hydraulic  Advanced  Low-Cost  G-Cueing  System 
(ALCOGS)  was  under  development  but  was  not 
available  for  use  In  the  ASPT  F-16  project. (3) 
However,  an  Improved  G-seat,  perhaps  capable  of 
onset  cueing,  was  felt  to  be  essential  to 
satisfy  the  motion  and  force  cueing  requirements 
of  high  performance  F-16  aircraft. 

APPROACH 

Provision  of  a  high  performance  G-cueing 
capability  for  the  ASPT  was  divided  Into  three 
phases: 

Phase  I:  The  objective  in  this  phase  was  to 
establish  the  nature  of  the  response  lags  in  the 
pneumatic  G-seat  system  and  to  minimize  those 
lags  by  optimizing  the  hardware  configuration 
and  the  software  execution.  Another  objective 
was  to  provide  a  more  usable  and  understandable 
program  which  would  facilitate  determination  of 
the  most  effective  drive  techniques  under  the 
following  phases. 

Phase  II:  The  objective  in  this  phase  was  to 
determine  the  most  effective  software  drive 
techniques  and  philosophies  under  the  optimized 
response  condition  of  Phase  I. 

Phase  III:  The  objective  in  this  phase  was  to 
attach  and  evaluate  new  G-cueing  devices  (such 
as  air  bladders  based  on  the  Advanced  low-cost 
G-cueing  design)  and  develop  new  sustained  drive 
techniques.  The  Improved  response  time  of  Phase 
I  also  made  feasible  the  development  of  special 
effects,  such  as  runway  rumble,  stall,  constant, 
airframe  vibration,  gear  effect,  and  other 
buffets. 

This  paper  summarizes  the  Phase  II  and  III 
efforts.  Phase  I  is  fully  documented  in  Ref 
(1),  but  a  summary  of  the  hardware  optimization 
portion  of  Phase  I  results  are  that  the  response 
time  could  be  decreased  by  as  much  as  half  (350ms 
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to  150-200ms)  by  the  following  modifications: 

1)  Moving  control  valves  to  the  platform  to 
reduce  hose  length. 

2)  Using  larger  Inside  diameter  hoses. 

3)  Relocating  needle  valves  from  the  Input 
to  output  of  booster  valves. 

4)  Resetting  needle  valve  and  booster  valve 
controls. 

The  software  optimization  portion  of  Phase  1 
resulted  In  a  more  organized  and  flexible  program 
which  was  used  to  vary  the  drive  philosophies  of 
each  G-seat  component.  For  the  Phase  II  and 
III,  the  ASPT  G-seat  was  considered  to  contain 
the  following  components:  seatpan,  backrest, 
thigh  panels,  lapbelt,  and  air  bladders; 

Responding  to  the  following  drive  Inputs: 

X-axis  acceleration 

Y-axis  acceleration 

Z-axIs  acceleration 

roll  velocity  or  roll  acceleration 

special  effects  conditions; 

Under  the  following  concepts: 

Translation:  The  elevation  of  a  complete  set  of 
cells  caused  to  translate  in  unison  a  uniform 
distance. 

Orientation  or  Attitude:  The  plane  formed  by  a 
complete  set  of  cells  Is  caused  to  be  reoriented. 

Contouring:  A  set  of  cells  caused  to  assume  a 
contoured  shape  which  produces  a  flesh  pressure 
redistribution  thought  to  be  compatible  with 
body  response  to  acceleration. 


Special  Effects:  Bumps,  vibrations,  buffets,  or 
other  special  responses. 


♦OPT  +  OPT  ♦  OPT  +  OPT  ♦  OPT  +  OPT 


t  «  translation 
a  •  attitude 
c  *  contouring 

Options  Included:  Acceleration  distortion, 
combinations,  modification  of  basic  logic, 
deadband,  etc. 

FIGURE  1  -  Research  Approach 


Attenuators  were  placed  upon  each  axis 
response;  for  each  drive  concept;  and  for  each 
component,  except  where  clearly  not  applicable 
(e.g.  contouring  for  lapbelt).  The  attenuators 
were  then  used  to  increase,  decrease,  or  reverse 
any  specific  response  based  on  axis,  drive 
concept  or  component. 

Due  to  the  numerous  possible  combinations  of 
accelerations,  axes,  drive  concepts  and  com¬ 
ponents,  It  was  decided  to  use  subjective  pilot 
feedback  as  the  only  criteria  for  developing  the 
drive  techniques.  In  addition  to  the  convenience 
of  this  method.  It  was  felt  that  any  other 
analysis  would  be  too  time-consuming  and 
eventually  subject  to  pilot  approval  as  the 
final  judge  of  Its  effectiveness. 

All  components,  drive  concepts,  or  axes  not 
currently  being  Investigated  were  deactivated  in 
order  to  Isolate  the  effects.  Once  a  valid 
response  was  developed,  another  axis,  drive 
concept,  or  component  was  added  and  adjusted 
until  it  either  contributed  to  the  effect  or  Its 
possibilities  were  exhausted. 

In  conjunction  with  this  basic  response 
Investigation,  all  other  options  of  the  driving 
philosophy  were  exercised  such  as  the  distortion 
logic  (non-linear  response),  deadbands,  combi¬ 
nations  of  accelerations,  and  occasionally 
modification  of  the  basic  controlling  software 
logic  or  addition  of  options.  If  an  Improvement 
was  not  determined  and  the  drive  did  not  present 
a  negative  cue,  it  was  returned  to  Its  original 
state  or  value.  The  final  product  thus  contains 
some  Improved  and  effective  drive  logic,  some 
original  and  effective  drive  logics,  and  some 
original  ineffective  drive  logic.  It  was  felt 
that  selective  deletion  of  ineffective 
capabilities  for  certain  axes  or  components 
would  not  significantly  Improve  the  G-seat  and 
would  tend  to  make  It  less  flexible  for  the 
Phase  III  research. 

PHASE  II  -  DRIVE  PHILOSOPHIES 

Seatpan 

The  firs?  component  Investigated  was  the 
seatpan.  Translation  response  in  the  Z  axis  was 
experienced  by  the  pilot  for  both  negative  and 
positive  G's.  The  computer  program  uses 
negative  G's  as  an  Indication  of  pullouts  and 
positive  G's  as  an  Indication  of  pushover  (this 
Is  the  opposite  of  the  normal  pilot  terminology). 
The  usual  response  of  seatpan  deflation  on 
pullouts  (position  cue  to  simulate  sinking  Into 
the  seat)  was  valid,  but  the  pushover  response 
was  preferred  reversed  so  that  the  bellows  also 
deflated  upon  pushover  for  a  correct  pressure 
cue.  The  pilot  felt  that  when  this  was  coupled 
with  a  relaxation  of  the  lapbelt,  an  Initial 
floating  effect  was  generated.  Thus,  the  seat- 
pan  has  the  same  Initial  response  (deflation) 
whether  positive  or  negative  G's  are  Imparted. 
The  difference  Is  In  lapbelt  response:  contrac¬ 
tion  on  negative  G's;  relaxation  on  positive  G's 
(pressure  cues).  At  approximately  zero  G's,  the 
belt  drive  reverts  to  that  of  contracting  upon 
additional  positive  G's  (position  cue).  The 
Z-axIs  contouring  arrays  (to  simulate  pressure 
distribution  changes)  were  also  modified  to 
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LOOSE 


UP 


RETURN  TO 
POSITION  CUES 


correspond  with  the  new  philosophy. 

Attitudinal  response  in  each  axis  drive  mode 
and  option  was  experienced  and  modified  if 
necessary.  The  most  significant  result  is  that 
a  plannar  response  to  roll  cues  (based  on  a 
driving  input  of  roll  velocity  rather  than 
acceleration)  which  simulated  a  tilting  plane 
was  effective  rather  than  using  the  contouring 
arrays  as  originally  designed.  In  the  contour¬ 
ing  concept  some  bellows  do  not  deflate  to  the 
same  degree  as  others  causing  a  localized 
pressure  increase  or  decrease  (redistribution  of 
body  weight).  Y-axis  contouring  was  consequently 
not  used. 

The  Y-axis  attitude  response  to  Y  accelera¬ 
tion  as  a  driving  input  should  vary  In  magnitude, 
depending  on  whetner  the  aircraft  is  on  the 
ground  or  in  the  air.  However,  if  a  roll 
condition  exists,  the  Y  acceleration  attitude 
drive  is  disabled  (l.e.,  an  attitude  response  to 
Y  acceleration  exists  only  if  an  attitude 
response  is  not  required  for  a  roll  cue). 

Lapbelt 

The  normal  drive  of  the  lapbelt  is  to 
contract  on  pushover  and  release  on  pullout  (to 
simulate  the  position  cue  of  the  pilot  sinking 
away  from  belt).  Upon  reversal,  however,  the 
lapbelt  tended  to  enhance  the  sinking  into  seat 
sensation  by  Increasing  the  buttock  pressure 
Imparted  to  the  pilot.  The  lapbelt  apparently 
distributed  the  pressure  over  a  wide  enough  area 
so  that  the  main  effect  was  felt  through  the 
seat-buttock  Interface  rather  than  the  belt- 
abdomen.  It  was  also  confirmed  that  the 
addition  of  a  G-suit  system  would  aid  in 
distributing  any  localized  lapbelt  pressure. 
Figure  2  shows  the  original  and  optimized 
response  of  the  lapbelt  and  seatpan  for  pushover 
and  pullout  maneuvers.  Note  that  the  seat  drive 
reverses  philosophy  at  the  1-G  level,  while  the 
lapbelt  does  not  reverse  until  the  zero-G  level. 
Software  was  installed  to  vary  the  slope  before. 


after,  and  within  the  "floating  effect"  area  and 
to  vary  the  bounds  of  each  area  for  each 
component.  This  was  Implemented  by  the 
following  technique  for  the  seatpan,  lapbelt, 
and  contouring. 

IF(Z.GT.1G  and  Z.LT.P2)  SL0PE=A 
IF(Z.LE.IG)  SLOPED 
IF ( Z . GE .  P2 )  SLOPED 

IF(Z.GE.P2)  CORRECT I0N=A*( P2 -1 G) -C*( P2-1G) 
RESPONSE*SLOPE*Z+CORRECTION 

Where: 

IG  :  386  in/sec^  (neutral) 

P2  :  Acceleration  levels  where  drive 

parameters  (slope)  may  change  values 

CORRECTION:  Necessary  to  maintain  continuity 
when  switching  parameter  values 
since  excursions  are  based  on  total 
acceleration  (sustained  drive  type) 
rather  than  deltas 

Below  neutral  1G  (pullout  maneuver)  the 
seatpan  deflates  to  create  a  sinking  sensation 
(position  cue)  but  the  belt  contracts  to 
Increase  perceived  pressure  cues.  Between 
neutral  (1G)  and  zero  G,  the  seatpan  and  the 
lapbelt  apply  pressure  (l.e.  less  pressure)  cues 
In  order  to  simulate  the  important  sensation  of 
floating;  the  seatpan  deflates  to  also  decrease 
pressure  and  the  lapbelt  relaxes  to  decrease 
pressure.  The  conflicting  position  cue  of 
sinking  into  the  seat  is  apparently  overriden  by 
the  pressure  cue  of  the  floating  sensation. 

Beyond  a  certain  acceleration  level  (zero-G)  the 
lapbelt  returns  to  providing  a  position  cue 
(pilot  floating  into  belt)  and  the  seatpan 

returns  to  a  position  cue  (raising  pilot  out  of 
seat).  The  conflicting  position  and  pressure 
cues  here  (pressure  from  rising  seatpan;  posi¬ 
tion  from  lapbelt)  is  minimized  by  a  reduction 
in  the  slope  of  response.  In  addition,  this 
flight  regime  is  seldom  experienced  for  any 
length  of  time. 
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»m=  TIME  THAT  MANEUVER  REACHES  FINAL  G  LEVEL 

t0=  TIME  THAT  SUIT  REACHES  PROPER  INFLATION  FOR  TERMINAL  G  LEVEL 


Figure  3  -  G-suit  Optimization 


Backrest 

The  backrest  response  to  y  acceleration  was 
also  changed  to  a  planar  type  and  the  y-contour- 
ing  deleted.  In  addition,  the  best  parameter 
values  for  response  on  the  ground  was  found  to 
be  different  from  those  in  the  air.  This  need 
was  satisfied  by  basing  the  change  In  parameter 
values  on  whether  there  was  weight  on  whec’s 
(MOW)  and  an  X  acceleration.  The  value  was 
replaced  to  a  ground  value  whenever  ViOVt  was 
"true"  which  tended  to  provide  a  touchdown  bunp 
as  the  cells  responded  to  the  new  parameter. 
However,  a  similar  but  Inappropriate  bump  would 
occur  on  takeoff.  The  switch  from  ground  to  air 
values  occur  only  when  MOW  Is  "false"  and  the  X 
acceleration  Is  near  zero.  In  this  case  no  bunp 
will  occur  upon  switchover  since  the  cells  will 
be  at  their  neutral  position. 

Other  Modifications 

Roll  velocity  was  selected  as  preferable 
over  roll  acceleration  as  a  driving  Input. 
Although  a  few  exceptions  were  noted,  the  G-seat 
performed  better  with  a  roll  velocity  Input  for 
the  majority  of  typical  maneuvers. 

Deadbands  were  preferred  on  the  X  axis  roll 
velocity,  and  roll  acceleration  drive  Inputs 
which  were  Independently  adjustable  for  positive 
or  negative  magnitudes. 


PHASE  III  -  HEW  DEVICES.  SPECIAL  EFFECTS  AND  NEW 
DRIVE  TECHNIQUES 

G-Suit 

A  G-suit  system  was  added  to  the  F-16 
simulation,  but  pilots  occasionally  commented 
about  a  relatively  low  rate  of  G-suit  inflation. 
This  led  to  the  following  lead  logic  scheme  for 
improving  the  typical  inflation  without  Incur¬ 
ring  over-inflation  In  maneuvers  of  higher 
G-  levels. 

The  lead  function  Is  to  cownand  the 
controlling  valve  to  open  wider  and  thus  Inflate 
the  suit  higher  In  a  shorter  time.  The  D/A  lead 
needs  to  be  a  function  of  not  only  the  G-level 
of  the  aircraft,  but  also  of  the  time  that  the 
G-level  has  not  been  achieved;  l.e. ,  the  command 
should  bleed  off  rapidly  enough  so  that  the  suit 
does  not  actually  overshoot  the  desired 
Inflation  level.  A  deadband  Is  also  necessary 
to  prevent  Instability.  The  lead  logic  would 
otherwise  tend  to  be  frequently  triggering  and 
never  bleed  off  conpletely. 

The  D/A  Initial  lead  conmand  Is  limited  for 
triggering  at  higher  G-levels  so  as  to  not 
actually  overshoot  desired  Inflation  levels. 

The  G-suit  logic  Is  deactivated  when  landing 
(gear  down)  to  prevent  Inappropriate  triggering. 


Thigh  Panels 

Similar  Investigations  for  the  thigh  panels 
component  were  generally  unsuccessful  In  that  a 
drive  philosophy  could  not  be  developed  which 
was  acceptable.  The  thigh  panel  did  not  provide 
any  cue  analogous  to  actual  flight  maneuvers. 

As  a  result,  the  thigh  panels  were  deactivated. 
This  also  made  air  sources  conveniently  available 
for  attachment  of  devices  under  Phase  III. 


D/A 

INITIAL 

LEAP 

COMMAND 


Figure  4  -  Lead  Control 
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The  slope  of  the  lead  factor  vs  G-level  was 
made  variable  along  with  the  limits  in  order  to 
determine  the  proper  constraints.  Lap  belt  and 
seatpan  parameters  were  also  adjusted  to 
complement  the  g-sult  cues. 

Helmet  Loader 


In  high  performance  aircraft,  the  G  forces 
on  the  pilot's  helmet  provide  Important  feedback 
concerning  the  aircraft's  dynamic  state  as  well 
as  limiting  the  pilot's  ability  to  move  his  head 
when  the  G  forces  are  high.  A  helmet  loader  has 
been  designed  by  Mr  Bill  Ashworth  and  Mr  Alton 
Hall  of  NASA/Langley  to  provide  the  effects  of 
these  forces  In  aircraft  simulators. (z)  The 
design  is  such  that  the  forces  applied  to  the 
helmet  are  independent  of  the  pilot's  head/ 
shoulder  position  and  require  only  two  strings 
to  be  attached  to  the  helmet  through  pulleys 
attached  to  the  shoulder  harness.  A  secondary 
effect  is  provided  through  a  loosening  of  the 
harness  straps  as  force  Is  exerted  downward  on 
the  helmet  (thus  upward  on  the  pulleys)  for 
positive  G.  A  prototype  helmet  loader  was 
loaned  from  NASA/Langley  for  use  In  the 
ASPT/F-16  project. 

The  pilots  did  not  have  access  to  their  own 
helmets  during  the  studies  which  probably 
contributed  highly  to  Individualistic  and 
conflicting  ratings  of  the  helmet  loader.  A 
summary  of  pilot  comments  are: 

-  Rotation  of  head  restricted  under  g 
loading  (although  some  thought  this  was 
realistic. 

-  Otherwise  no  restriction  of  normal 
movements 

-  Could  positively  affect  performance 
(based  on  opinions) 

-  Single  most  overriding  comment:  Not 
enough  force  (approx  10  lbs) 

Thus  a  system  capable  of  safely  applying 
higher  forces,  without  restricting  head 
rotation,  and  permitting  use  of  personal  helmets 
may  be  an  effective  hlgh-g  cueing  device. 

Air  Bladders/Plates 

Air  bladders  and  overlay  plates  had  been 
manufactured  during  the  Advanced  Low-Cost 
G-Cuelng  System  (ALCOGS)  development  and  used  to 
simulate  the  ALCOGS  design  on  the  penumatlc  ASPT 
g-seat. I1)  mis  research  had  established  that 
the  bladders  and  plates  provided  a  more 
integrated,  continuous  sensation  of  a  seat 
structure  than  the  original  ASPT  design. 

Due  to  the  Ineffectiveness  of  ASPT's  thigh 
panels,  the  panels  were  removed  and  4  unit  air 
bladders  for  the  seatpan  and  a  2-unit  backrest 
were  attached  to  those  hoses.  The  control  valve 
settings  were  adjusted  to  reduce  the  pressure 
range  from  0-6  psl  to  approximately  0-2  psl.  An 
overlay  plate  was  placed  over  the  16  seatpan 
cells.  Thigh  wedges  and  tuberosity  blocks  were 
Installed  on  the  plate  to  provide  the  contouring 
concept.  The  air  bladder  was  Installed  over  the 
entire  assembly  with  layers  of  foam  separating 
each  level  (between  bellows  and  plate,  plate  and 
bladder,  bladder  and  seat  cover). 


Air  bladder 
(Partially  exploded) 
Overlay  plate 


JL. 


□  □  □□ 


—-Foam 


G-seat  bellows 


Figure  5  -  ALC06S  Concept 


The  relevant  thigh  panel  longitude,  lateral, 
and  vertical  drives  were  altered  to  reflect 
their  use  as  the  appropriate  bladder  driving 
logic. 

A  weight  bias  was  used  in  order  to  Insure 
that  the  pilots  tuberosities  rest  just  above  the 
plate  at  neutral.  Otherwise,  a  conflicting  cue 
may  exist  upon  deflation  and  before  contact  with 
the  plate. 

The  air  bladders  act  supplementally  to  the 
orientation  drive  and  translational  drive  of  the 
normal  ASPT  seat  replacing  the  pressure 
redistribution  function  of  "contouring"  and 
provide  some  position  cueing  as  well.  The 
previous  contouring  drive  was  deactivated  since 
the  overlay  plate  prevented  the  pilot  from 
contacting  the  Individual  bellows  of  the  ASPT 
G-seat. 


In  case  of  a  pullout,  the  seatpan  now 
deflates  to  supply  a  position  cue;  all  four 
units  of  air  bladder  deflate  to  bring  the 
buttocks  Into  contact  with  the  thigh  panels  and 
tuberosity  blocks;  and  the  lapbelt  contracts  to 
enhance  the  Increased  pressure  effect.  In  a 
roll  the  seatpan  acts  as  a  plane  and  rotates, 
and  one  side  of  the  bladder  deflates  to  increase 
pressure  sensation  on  the  appropriate  thigh. 


Y  ACCEL  on  ROLL 
Figure  6  -  Air  Bladder  Use 


Special  Effects 

In  the  optimized  configuration  the  ASPT 
g-seat  was  capable  of  providing  a  coordinated 
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vibration  effect  up  to  at  least  IS  Hz.  Although 
It  could  accomplish  only  a  small  portion  of  the 
commanded  magnitude  (l.e.  15  Hz  was  well  beyond 
Its  bandpass  capacity  of  approximately  2-3  Hz) 
the  existing  excursions  were  more  than 
sufficient  to  provide  satisfactory  special 
effects  and  in  some  cases  had  to  be  reduced  from 
the  maximum  possible. 

A  general  vibration  logic  was  developed  as 
follows: 

Basic  Magnitude  (MAG)  Calculations 


Attitude  response  sensitivity  was  made  a 
function  of  landing  mode  (gear  down)  where 
higher  sensitivity  was  desired  as  a  flare/ 
landing  cue.  Similarly,  the  deadbands  were 
decreased  and  special  effects  generated  (gear 
buffet,  bump)  based  on  whether  the  gear  was  "not 
up  and  locked."  The  deadbands  were  also  based 
upon  whether  a  positive  or  negative  acceleration 
maneuver  was  being  performed.  The  following 
table  summarizes  the  sensitivity  of  components 
or  effects  based  on  flight  conditions. 


C0UNTER-C0UNTER+1 

IF  ( COUNTER. LE. FLAG)  MAG=MAG+VIBMAG 

IF  ( COUNTER. GT. FLAG)  MAG=MAG-VI8MAG 

MAX»2.*FLAG+1. 

IF  ( COUNTER. GE. MAX)  COUNTERS. 

The  counter  Is  Incremented  upon  each  computer 
pass  (30  times  per  second).  The  magnitude  was 
adjustable  through  VIBMAG  and  the  frequency 
adjustable  through  FLAG  as  follows: 

FLAG 
0 
1 
2 

3 

4 
6 

Where  FREQ*15/( FLAG+1 ) ;  15  Is  the  max 
frequency,  derived  from  the  computational  rate 
of  30  iterations  per  second  and  2  iterations 
reguired  for  one  complete  cycle. 

The  vibrational  frequency  and  magnitude  was 
varied  throughout  a  wide  range  to  determine  the 
best  technique  for  simulation  of  constant 
airframe  vibration,  speedbrake,  gear  effects 
(down,  in-transit,  and  bump),  runway  rumble,  and 
stall. 

The  results  were  as  follows: 


FREQ 

Constant  Airframe  Vibration  15.0  Hz 

Speedbrake  Buffet  5.0  Hz 

Runway  Rumble  (varies)  7.5  Hz 

Gear  Bump  NA 

Gear  Down  Rumble  15.0  Hz 

Gear  In  Transit  Rumble  15.0  Hz 

Stall  Buffet  3.0  Hz 


These  results  were  specific  to  this 
simulation,  choice  was  limited  to  the  available 
frequencies  and  could  vary  based  on  the  aircraft 
simulated  and  the  system  characteristics. 

Thus,  at  least  a  30  Hz  Iteration  rate  Is 
desirable  for  simulation  of  speedbrake  buffet 
(15/3),  constant  airframe  vibration  (15/1),  gear 
down  rumble  (15/1),  and  gear  In  transit  rumble 
(15/1). 


FREQ 
1575)  Hz 

7.50  Hz 
5.00  Hz 
3.75  Hz 
3.00  Hz 

2.50  Hz 


G/A 

UNDING 

G-LEVEL 

ROLL 

Seatpan 

X 

X 

X 

X 

Backrest 

X 

X 

X 

X 

Lapbelt 

X 

G-sult 

X 

X 

X 

Bladders 

X 

X 

X 

X 

Deadbands 

X 

X 

X 

Special  Effects 

X 

X 

Conclusions 


A  pneumatic  g-seat  can  be  effective  as  a 
motion/force  cueing  device  for  high  g,  high 
performance  aircraft  provided  the  response  time 
is  optimized  over  the  typical  configuration, 
more  effective  components/concepts  are  used,  and 
the  driving  philosophy  somewhat  dynamic  based  on 
the  flight  conditions  or  maneuvers  being 
performed. 


Additional  Improvements  In  response  time  and 
bandpass  can  probably  be  achieved  only  through  a 
design  change  In  the  controlling  valves  and/or 
closing  the  system  with  feedback  (position 
preferred  over  pressure  since  proper  excursion 
Is  the  objective);  but  a  fully  optimized 
pneumatic  g-seat  may  have  the  potential  to 
provide  effective  onset  cues  (currently 
attempted  by  6  DOF  motion  system)  as  the  most 
responsive  of  the  force-cueing  devices. 
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ABSTRACT 


The  early  use  of  special  purpose  analog  computers  in  flight  trainers  faced  many  problems. 

These  difficulties  along  with  the  recognition  that  the  then  emerging  digital  computer  tech¬ 
nology  might  be  applied  to  training  systems  led  to  investigations  of  this  area.  The  purpose  of 
this  paper  is  to  trace  the  research  and  development  of  high-speed  digital  computers  for  real¬ 
time  operations  in  solving  the  flight  equations  in  real  time  through  the  current  state-of-the- 
art  digital  systems  used  in  flight  trainers.  Three  basic  aspects  of  real-time  digital  computer 
systems  for  flight  trainers  are  reviewed;  namely,  computer  system  architecture,  iteration  rates 
required,  and  programming.  In  addition,  a  forecast  of  future  systems  architecture  and  program¬ 
ming  concepts  based  on  research  currently  underway  is  offered. 


INTRODUCTION 

Early  attempts  made  to  utilize  digital  compu¬ 
ter  technology  in  the  solution  of  flight  equations 
in  real  time  faced  many  problems.  The  purpose  of 
this  paper  is  to  trace  the  research  and  development 
of  flight  trainers  utilizing  high-speed  digital 
computers  for  real-time  operations  in  solving  the 
flight  equations  through  the  current  state-of-the- 
art  digital  systems  used  in  flight  trainers.  Three 
basic  aspects  of  real-time  digital  computer  systems 
for  flight  trainers  will  be  reviewed;  namely,  com¬ 
puter  system  architecture,  iteration  rates,  and 
progranming.  In  addition,  a  forecast  of  future 
systems  architecture  and  programming  concepts  based 
on  research  currently  underway  is  offered. 

INITIAL  CONCEPTS  AND  DEVELOPMENTS 

Inmediately  following  World  War  II,  the  Office 
of  Naval  Research,  then  the  Office  of  Research  and 
Invention,  undertook  the  development  of  a  number  of 
computer  systems  (1).  The  Whirlwind  system  was  a 
digital  computer  development  whose  initial  objec¬ 
tive  was  to  simulate  the  flight  envelope  of  an  air¬ 
craft  to  obtain  pilot  evaluation  of  proposed  air¬ 
craft  based  upon  wind  tunnel  data  before  going  to 
the  expense  of  developing  a  prototype  (2).  Although 
it  did  not  meet  its  basic  objective,  it  did  result 
in  a  very  powerful  computer  for  its  day.  However, 
the  scientific  and  engineering  applications  of  the 
digital  computer  led  to  a  more  rigorous  mathemati¬ 
cal  description  of  the  equations  of  flight. 

The  flight  trainers  developed  by  the  Armed 
Services  in  the  late  forties  called  for  the  solu¬ 
tion  of  the  differential  equations  of  motion  using 
analog  computer  systems  (3)  (4).  However,  diffi¬ 
culties  were  experienced  in  the  development  and  use 
of  analog  computer  systems  for  flight  trainers. 
First,  the  design  of  the  computer  itself  was  con¬ 
strained  by  the  availability  and  accuracy  of  aero¬ 
dynamics  and  system  data  on  the  aircraft.  Second, 
once  the  system  was  built  and  testing  revealed 
errors  in  the  aerodynamic  data,  the  computer  hard¬ 
ware  design  had  to  be  changed.  Third,  the  errors 
and  limitations  of  analog  systems  led  to  serious 
acceptability  issues  with  users.  These  difficulties 
motivated  the  exploration  of  digital  technology. 

The  expected  advantages  of  this  technology  were  (a) 
that  programming  would  be  Independent  of  the  com¬ 


puter  and  input-output  system  design  and  (b) 
changes  in  aircraft  aerodynamic  data  could  be 
incorporated  without  hardware  changes  to  the 
computer  system  in  most  cases. 

The  failure  of  Whirlwind  as  a  real-time 
digital  computer  for  application  to  flight  simula¬ 
tion  triggered  an  investigation  into  the  reasons 
for  the  failure  and  led  to  the  research  necessary 
to  develop  the  required  technology.  This  study 
was  initiated  in  1949  and  continued  through  1955. 

It  was  conducted  by  the  Moore  School  of  Engineering 
at  the  University  of  Pennsylvania  under  the  spon¬ 
sorship  of  the  Navy.  The  results  of  the  early 
study  in  1950  (5)  concluded  that  there  did  not 
exist  a  digital  computer  system  capable  of  solving 
the  flight  equations  for  one  airplane  in  real  time 
using  integration  schemes  in  common  use  at  that 
time.  However,  later  studies  from  1952  to  1954 
concluded  that  the  real-time  simulation  of  aircraft 
using  a  digital  computer  was  feasible  (6).  The 
basis  for  this  conclusion  was  the  development  of 
(a)  high-speed  random  access  memory  and  digital 
computer  logic  which  increased  the  speed  of  compu¬ 
tation,  (b)  new  methods  of  numerical  integration 
which  were  consistent  with  the  speed  of  the  then 
current  digital  computers,  and  (c)  time-efficient 
methods  of  function  generation  for  calculating 
aerodynamic  coefficients  and  stability  derivatives. 

In  1956  a  contract  was  awarded  under  the  spon¬ 
sorship  of  the  Navy  and  the  Air  Force  for  the 
design,  development,  and  construction  of  a  digital 
computer  system  in  accordance  with  the  logical 
computer  structure  and  circuits  designed  and 
developed  by  the  Moore  School.  This  development 
effort  came  to  be  known  as  the  Universal  Digital 
Operational  Flight  Trainer  (UDOFT)  (7).  The  purpose 
of  this  effort  was  to  demonstrate  that  a  digital 
computer  system  could  be  designed  to  simulate  in 
real  time  subsonic  and  supersonic  jet  aircraft 
(F9F-2  and  F-100A  respectively)  utilizing  the  appro¬ 
priate  cockpit  configurations  and  an  associated 
instructor  control  station.  Benefits  derived  from 
the  use  of  a  digital  computer  system  were  improved 
accuracy,  flexibility,  and  the  relative  ease  of 
programming.  The  computational  capability  of  the 
UDOFT  computer  system  is  provided  in  Table  I.  Table 
I  also  summarizes  the  essential  characteristics  of 
digital  computers  used  in  flight  trainers  keyed  to 
the  time  of  development. 
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TABLE  I.  FLIGHT  TRAINER  COMPUTER  SYSTEM  CHARACTERISTICS 


F/A-18  2F132  1979  4  -  SEL  32/77  600  NSEC  128  KW  EACH  MOS  2-300  FORTRAN 

OFT  64  KW  SHARED  MB  DISC 


COMPUTER  SYSTEM  ARCHITECTURE 


complex. 


The  computer  design  and  mathematical  develop¬ 
ments  which  led  to  the  initial  operation  of  UDOFT 
had  a  significant  effect  upon  the  computer  communi¬ 
ty.  However,  with  the  invention  of  the  transistor 
and  the  rapid  acceptance  of  the  digital  computer 
in  the  commercial  world  of  the  late  fifties  and 
sixties,  the  available  computational  power  in¬ 
creased  by  an  order  of  magnitude  and  the  cost  of 
digital  hardware  decreased.  The  effects  of  these 
trends  can  be  clearly  traced  from  the  UDOFT  base¬ 
line  to  current  day  systems. 

Let  us  consider  the  area  of  computer  system 
architecture  as  it  was  applied  to  real-time  flight 
trainers.  The  early  constraints  of  minimal  memory 
availability  and  arithmetic  processing  capability 
coupled  with  high  cost  generally  led  to  a  design 
strategy  of  performing  only  the  basic  aircraft  and 
related  systems  modeling  in  the  digital  computer. 

As  a  result  many  of  the  input  and  output  operations 
were  performed  in  the  interface  unit,  such  as  for¬ 
matting  for  data  entry,  digital  to  voltage  and 
shaft  position  conversion  and  binary  to  decimal 
conversion.  The  technology  was  generally  that  of 
discrete  component  circuit  boards. 

An  example  is  the  UDOFT  system.  Its  random 
access  memory  size  was  8192  20-bit  words  with  a 
magnetic  tape  utilized  for  external  storage  of 
programs  and  data.  A  block  diagram  of  the  computer 
system  is  shown  in  figure  1.  The  central  proces¬ 
sing  unit  (CPU)  was  specially  designed  to  perform 
the  numerical  calculations  necessary  (function 
generation,  integration,  coordinate  transforma¬ 
tions,  etc.)  for  simulation  of  aircraft.  The  F9F- 
2  or  F-100A  jet  fighters  were  the  aircraft  types 
selected  to  demonstrate  feasibility. 

The  first  flight  trainer  development  which 
utilized  a  conmercially  available  general  purpose 
digital  computer  in  its  computer  subsystem  was 
Device  2F84  A7A  Weapon  System  Trainer.  This  flight 
trainer  employed  a  single  DDP-224  computer.  A 
block  diagram  of  this  computer  system  is  shown  in 
figure  2.  The  design  was  a  direct  outgrowth  of  the 
research  efforts  on  UDOFT.  The  one  major  goal, 
namely  that  of  using  a  conmercially  developed 
computer  was  successfully  accomplished.  Another 
major  research  goal,  that  of  separating  computer 
development  from  trainer  hardware  development,  was 
also  achieved.  This  achievement  permitted  the 
initial  aerodynamic  and  stability  data,  which  are 
usually  Inaccurate  or  incomplete,  to  be  updated 
through  reprogramming. 

As  computer  technology  advanced,  the  computer 
system  designs  of  flight  trainer  systems  also  un¬ 
derwent  change.  Device  2F90,  TA4J  Operational 
Flight  Trainer,  used  a  multi -processor  system 
architecture  (8).  In  this  system  two  CPU's  shared 
the  computational  load  for  four  simulated  TA-4J 
aircrafts.  A  block  diagram  of  the  computer  system 
is  shown  in  figure  3.  The  computations  were 
allocated  to  each  of  the  CPU's  by  function.  As  a 
result  both  CPU's  were  required  for  system  opera¬ 
tion.  The  advantage  of  this  system  was  the  comput¬ 
ing  capability  possessed  by  nature  of  the  CPU's 
operating  simultaneously;  Its  disadvantage  was  that 
it  had  undesirable  reliability  characteristics  in 
that  if  one  CPU  failed,  the  system  became  Inopera- 
tlve.  It  also  complicated  the  programming  task 
and  made  manufacturing  and  engineering  tests  more 


Experience  from  this  system  led  to  the  design 
concept  of  distributed  processing.  The  computa¬ 
tional  load  was  divided  into  its  lowest  common 
denominator;  i.e.,  one  CPU  per  cockpit.  This  con¬ 
cept  had  the  advantage  of  parallel  processing  to 
conserve  time,  ease  of  manufacturing  and  checkout, 
and  system  reliability.  It  also  permitted  the  num¬ 
ber  of  trainee  stations  to  be  reduced  or  expanded 
without  a  change  to  the  basic  system  design.  The 
2F101,  T2C  Operational  Flight  Trainer,  consisted 
of  four  CPU's  driving  four  cockpits  (Figure  4). 

Each  individual  computer  system  provided  the  compu¬ 
tational  power  for  calculating  the  flight  simulation 
(equations  of  flight),  the  aircraft  system,  the 
Instructor  station,  and  performing  the  operations 
of  the  executive  system.  Note  that  more  and  more 
functions  are  being  assumed  by  the  digital  computer. 

The  development  of  Device  2F131,  A6E  Opera¬ 
tional  Flight  Trainer,  and  2F132,  F18A  Operational 
Flight  Trainer,  represented  a  major  turning  point 
in  the  application  of  digital  computer  technology 
to  flight  trainers.  The  specifications  for  this 
trainer  required  simulation  of  a  rather  complex 
aircraft,  the  inclusion  of  an  extensive  number  of 
malfunctions,  and  the  augmentation  of  problem 
control  and  scenario  generation  capability  through 
an  interactive  CRT  keyboard  system.  A  30-Hz  itera¬ 
tion  requirement  for  the  critical  flight  equations 
was  specified.  These  design  requirements  led  to 
a  computer  configuration  for  the  device  consisting 
of  four  CPU's  to  support  one  trainee  station.  The 
fivefold  increase  in  memory  requirements  over  pre¬ 
vious  systems  dramatized  the  impact  of  the  change 
in  functional  requirement  coupled  in  some  way  to 
the  decreases  in  the  cost  of  digital  computers. 
Computing  speed  and  memory  capacity  of  current  day 
systems  led  to  serious  software  design  and  valida¬ 
tion  problems  which  currently  exist.  These  soft¬ 
ware  problems  have  contributed  significantly  to  the 
lengthening  of  the  development  cycle.  The  advan¬ 
tages  of  hardware  redundancy,  manufacturability, 
life-cycle  support  costs,  and  programming  checkouts 
were  further  enhanced.  In  addition,  the  parallel 
processing  within  the  system  minimized  the  trans¬ 
port  delay  problem.  Figure  5  shows  a  block  diagram 
of  the  computer  system  for  Device  2F132. 

At  the  present  time,  research  is  being  con¬ 
ducted  in  the  area  of  distributed  processor  com¬ 
puter  system  architecture.  The  thrust  is  to 
further  extend  the  distributed  processor  concept 
or  aircraft  into  subdivisions  each  with  its  own 
mini-  or  microprocessors.  The  Increased  complexity 
of  aircraft  (for  example,  the  F18  of  today  versus 
the  F9  of  the  mid-fifties,  and  the  Increased  com¬ 
puting  power  of  mini -and  microcomputers)  have  made 
this  alternative  attractive  and  logical.  As  time 
has  passed,  more  and  more  of  the  simulation  func¬ 
tions  have  been  assuned  by  the  digital  system  por¬ 
tions  of  the  trainer  simulator.  Microprocessor 
technology  has  resulted  In  a  shift  in  the  location 
of  the  data  distribution  and  conversion  system 
from  the  computer  area  to  the  cockpit.  This  de¬ 
sign  minimizes  the  cockpit  to  computer  cabling 
requirement. 

ITERATION  RATES 

The  use  of  the  general-purpose  digital  compu¬ 
ter  for  obtaining  numerical  solutions  of  the 
flight  equations  in  real  time  raised  the  signifi- 


Figure  1.  UDOFT  Computer  System 
Block  Diagram 


Figure  2.  A7A  Computer  System  Block  Diagram 
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Figure  4.  2F101  Computer  System  Block  Diagram 
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Figure  5.  ZF132  Computer  System  Block  Diagram 
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cant  issue  of  the  iteration  rates  at  which  the 
equations  must  be  solved  to  ensure  stable  and  ac¬ 
curate  computation.  The  initial  program  designs 
used  the  heuristically  established  Iteration  rate 
of  20  Hz.  Since  the  outputs  from  the  digital  com¬ 
puter  driving  the  Instruments  were  discrete  values, 
an  '■■jtput  rate  fast  enough  to  create  the  illusion 
of  continuous  motion  was  necessary.  Motion  picture 
technology  was  the  basis  for  Initially  selecting 
the  20  Hz  iteration  rate.  Subsequent  investigations 
established  that  the  aircraft's  natural  frequencies 
were  in  the  range  of  0.75  to  1.0  Hz  and  varied 
directly  with  speed  and  inversely  with  altitude. 
Theoretical  studies  of  digital  sampled  data  systems 
have  determined  that  it  is  necessary  to  take  more 
than  two  points  per  cycle  of  the  highest  signifi¬ 
cant  frequency  component  in  a  signal  in  order  to 
recover  that  signal.  Hence,  at  20  samples/second, 
the  iteration  rate  was  approximately  10  times  that 
required  to  represent  the  ideal  aircraft  motion 
as  determined  by  Nyquist  rate. 

However,  other  factors  are  involved.  Other 
aircraft  systems  Influenced  the  selection  of  the 
iteration  rate.  With  one  exception,  the  natural 
frequency  of  aircraft  subsystems  did  not  exceed 
those  of  the  aircraft  itself.  The  exception  was 
the  flight  control  system.  The  pilot  has  the  abil¬ 
ity  to  introduce  a  step  input  into  the  system. 

Early  digital  systems  did  not  face  computational 
stability  problems  at  the  20  Hz  rate  because  of  the 
filtering  inherent  in  the  analog  control  system 
design.  Initial  attempts  to  convert  the  control 
input  directly  into  digital  signals  proved  disas¬ 
trous.  Preliminary  analysis  indicates  that  itera¬ 
tion  rates  in  excess  of  1000  Hz  may  be  necessary 
to  achieve  stable  computations  (9). 

As  system  complexity  Increased,  the  competi¬ 
tion  for  computational  time  also  increased.  This 
led  to  the  establishment  of  subdivisions  of  the 
highest  Iteration  rate  to  reduce  the  number  of 
computations.  Program  design  based  on  Iteration 
rate  subdivision  has  become  corrmon.  One  finds 
rates  of  16,  8  and  4  Hz,  for  example.  As  an  expe¬ 
dient,  the  computations  allocated  to  these  rates 
were  sometimes  shifted  to  lower  rates  when  running 
time  constraints  could  not  be  satisfied.  As  a  re¬ 
sult  of  these  trade-off  decisions,  serious  problems 
in  system  performance  with  man  in  the  loop  became 
evident. 

The  inability  to  develop  reasonable  estimates 
of  computer  capacity  required  for  a  device  has 
resulted  in  major  expansions  in  computer  capacity 
and  serious  programming  problems  during  develop¬ 
ment.  These  data  led  the  Navy  to  reassessment  of 
its  spare  requirements  for  time  and  memory.  The 
spare  running  time  reserve  requirement  was  In¬ 
creased  from  25%  to  50%.  Spare  memory  requirements 
were  increased  from  50%  to  100%.  The  specification 
change  was  implemented  in  1976.  The  primary  rea¬ 
son  for  increasing  this  requirement  was  to  ensure 
the  availability  of  spare  capacity  to  accomodate 
changes  during  the  life  cycle  of  the  device  (be¬ 
tween  15  and  20  years).  It  was  also  Intended  to 
provide  a  cushion  during  the  design  phase,  thereby 
avoiding  costly  computer  system  expansion  late  In 
the  development  program. 

The  transport  delay  problem  was  well  masked 
for  some  time  because  of  the  time  delay  Inherent 
in  most  Instruments  utilized  by  the  pilot  to  con¬ 
trol  the  aircraft.  The  fidelity  of  simulation 


issue,  which  has  always  been  questionable,  was  be¬ 
lieved  to  be  the  culprit.  With  the  advent  of 
visual  systems  attachments  for  flight  trainers,  the 
severity  of  the  transport  delay  problem  was  identi¬ 
fied  (10).  The  visual  system  horizon  reference 
acted  as  a  magnifying  glass  for  aircraft  attitude 
information.  Pilot  performance  in  the  simulator 
became  erratic.  Studies  were  conducted  to  determine 
the  acceptable  transport  delay  levels  (11).  As  a 
result  of  these  studies  and  the  requirement  to  min¬ 
imize  delays  caused  by  dissimilar  and  assynchronous 
Iteration  rates  between  the  30  Hz  television  frame 
rate  and  the  prevalent  16  Hz  flight  dynamics  update 
rate,  a  30  Hz  iteration  was  established  by  the  Navy. 

The  selection  of  the  30  Hz  basic  iteration 
rate  seemed  to  have  solved  the  problem.  But  this 
was  not  to  be  so.  The  current  generation  of  opera¬ 
tional  aircraft  have  general-purpose  computers  that 
perform  a  number  of  aircraft  operations  and  tactical 
functions.  If  this  on-board  computer  accepts  input 
at  a  given  rate,  then  it  would  be  logical  to  update 
that  information  at  its  design  rate.  In  some  cases 
the  rate  has  been  as  high  as  50  Hz.  This  is  proba¬ 
bly  much  higher  than  is  required  to  produce  stable 
and  accurate  solutions  of  the  flight  equations. 

PROGRAMING  LANGUAGE  AND  SOFTWARE  ENGINEERING 

The  initial  UWFT  effort  was  based  upon  the 
programing  strategy  developed  by  the  University  of 
Pennsylvania  (12).  The  strategy  was  basically 
machine-language  oriented.  Programni ng  was  geared 
to  saving  memory  space  and  achieving  execution 
time  efficiency.  The  executive  or  operating  system, 
such  as  it  was,  was  tailored  to  real-time  applica¬ 
tion.  The  time  frame  for  the  effort  was  in  the  late 
fifties.  In  addition,  the  real-time  needs  of  the 
system  had  essentially  dictated  the  computer  archi¬ 
tecture  and  Instruction  set  design. 

With  the  advent  of  powerful  general-purpose 
digital  computers  using  solid  state  design  in  the 
early  sixties,  the  need  for  more  effective  program¬ 
ming  tools  became  clearly  evident,  and  more  power¬ 
ful  assemblers  and  compilers  became  available. 
However,  for  real-time  simulation,  only  assembly 
language  was  utilized.  In  order  to  achieve  system 
efficiency  for  real-time  operation,  specially  de¬ 
signed  executives  were  still  required,  since  real¬ 
time  clocks  and  other  special  input/ output  devices 
had  to  be  accommodated  in  operating  system  software. 

As  computing  speed  continued  to  increase  and 
the  cost  of  computer  hardware  for  a  given  capability 
decreased,  assemblers  continued  to  be  utilized. 

There  was  a  gradual  shift  to  the  use  of  the  general- 
purpose  operating  system  provided  by  the  CPU  manu¬ 
facturer  instead  of  a  special  purpose  real-time 
executive.  There  was  a  good  reason  for  this  trend. 
The  high-speed  minicomputers  had  a  real-time  process 
control  market. 

As  computer  hardware  technology  continued  to 
advance,  the  low  cost  systems  became  more  prevalent 
and  very  cost  effective.  Those  developments  essen¬ 
tially  triggered  a  shift  of  emphasis  from  a  compu¬ 
ter  hardware  based  development  to  a  software  inten¬ 
sive  environment.  This  shift  had  a  major  impact  on 
trainer  development.  First,  it  caused  an  Increase 
in  the  number  of  functions  which  could  be  performed 
by  a  given  computing  system.  Second,  It  caused  a 
significant  Increase  in  the  amount  of  programing 
required  which,  in  turn,  led  to.  a  search  for  more 
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productive  methods  for  developing  software.  The 
use  of  higher  order  languages  (HOL's)  in  this  re¬ 
gard  became  of  interest.  However,  most  HOL's  were 
relatively  time  and  memory  inefficient  for  use  In 
real-time  systems.  This  led  to  a  need  for  in¬ 
creased  computer  capacity. 

The  feasibility  of  a  simulation  oriented, 
real-time  HOL  was  investigated  in  the  early  seven¬ 
ties.  Initial  response  was  that  existing  HOL's 
could  not  do  the  job  effectively.  Studies  were 
conducted  by  the  University  of  Florida  (13)  to 
specify  a  language  applicable  to  real-time  simula¬ 
tion  for  training  and  to  Identify  problem  areas 
which  needed  further  work.  The  Air  Force  also  con¬ 
ducted  a  study  into  the  applicability  of  FORTRAN 
for  real-time  simulation  (14). 

In  general,  these  studies  showed  that  applica¬ 
tion  of  HOL's  for  real-time  simulation  programming 
would  be  feasible,  but  that  additional  language 
features  and  run-time  efficiencies  would  be  re¬ 
quired  to  make  the  application  reasonably  effective. 
With  the  recognition  of  the  trends  in  software 
development  stated  above,  an  Engineering  Dept. 
Directive  was  issued  by  the  Naval  Training  Equip¬ 
ment  Center  in  1973.  This  Directive  established 
the  need  for  HOL's  and  identified  the  use  of 
FORTRAN  as  a  short  range  goal.  The  first  specifi¬ 
cation  to  require  the  use  of  FORTRAN  was  Specifi¬ 
cation  2222-1139A,  Specification  for  Air  Combat 
Manuevering  Simulator,  Device  2E6,  dated  16  August 
1976. 

Initially,  the  industry  reaction  to  this  re¬ 
quirement  was  negative.  The  desire  to  circumvent 
the  requirement  led  to  many  criticisms  of  FORTRAN, 
but  soon  this  resistance  waned  and  the  benefits  of 
FORTRAN  began  to  be  extolled.  Two  papers  (15,  16) 
presented  at  the  Naval  Training  Equipment  Center's 
Industry  Conferences  indicated  the  relative  effec¬ 
tiveness  of  FORTRAN.  Deficiencies  identified  in 
previous  studies  of  the  applicability  of  the  FOR¬ 
TRAN  compiler  were  restated.  The  relative  compiler 
efficiency  between  manufactures  was  also  identi¬ 
fied.  The  requirement  for  the  use  of  FORTRAN  is 
now  generally  accepted.  The  timing  for  this  accep¬ 
tance  was  perhaps  fortuitous.  The  Increasing  de¬ 
mand  for  instructional  capability  coupled  with  in¬ 
creased  system  complexity  has  caused  a  software 
engineering  explosion. 

It  would  be  unthinkable  to  discuss  computers 
and  real-time  programing  without  addressing  the 
critical  software  problem  which  faces  industry. 

The  software  explosion  is  dramatically  portrayed 
by  the  data  presented  in  figure  6.  Even  with  the 
low  cost  computer  systems  that  are  available  in 
the  market  place,  numerous  projects  have  substan¬ 
tially  exceeded  the  bid  stage  CPU  estimates  of 
both  run  time  and  memory  capacity.  The  sheer  size 
of  the  programming  effort  and  the  lack  of  adequate 
control  of  software  development  has  destroyed  the 
credibility  of  software  designers.  Program  and 
engineering  managers  are  eagerly  seeking  solutions 
to  this  problem,  a  problem  which  has  finally  re¬ 
ceived  their  attention.  The  government  has  taken 
the  initiative  by  requiring  a  more  vigorous  and 
disciplined  approach  to  software  development.  The 
Air  Force  has  adopted  Air  Force  Regulation  800-14. 
The  Navy  has  adopted  MILSTD  1644.  Efforts  are  un¬ 
derway  to  standardize  these  documents. 

Today,  the  trend  is  toward  the  adoption  of  ADA, 


the  DOD  HOL,  when  It  becomes  available.  In  fact, 
the  Computer  Systems  Laboratory  of  the  Naval  Train¬ 
ing  Equipment  Center  has  participated  In  the  Phase 
III  evaluation  of  ADA  for  a  trainer  real-time  simu¬ 
lation  application. 

In  addition  to  a  disciplined  approach  to  the 
development  process,  there  is  a  pressing  need  for 
more  information  about  the  characteristics  of  the 
program.  The  data  shown  in  Table  II  is  only  one 
way  to  show  the  structure  of  the  program.  Another, 
and  perhaps  more  meaningful  from  an  analysis  view¬ 
point,  would  be  a  breakdown  by  numerical  function, 
such  as  integration,  function  generation,  coordinate 
transformation,  searches,  logical  computation,  and 
others.  This  breakdown  is  subroutine  oriented. 

Such  data  would  reveal  more  about  what  the  program 
is  doing.  A  further  extension  of  this  breakdown 
would  be  a  count  of  the  nianber  of  times  each  instruc¬ 
tion  is  executed  in  a  program  cycle.  Why  this  need? 

A  search  of  the  literature  shows  little  documenta¬ 
tion  along  these  lines.  The  sketchy  nature  of  the 
data  shown  in  Figure  2  and  the  failure  of  a  litera¬ 
ture  search  to  show  any  data  on  real-time  program 
structure  support  this  position.  Efforts  are  being 
initiated  at  NAVTRAEQUIPCEN  in  the  Computer  Systems 
Laboratory  to  collect  such  data  as  an  adjunct  to 
computer  research  tasks.  We  encourage  Industry  to 
do  the  same  and  to  share  these  results  with  the 
training  device  community.  This  type  of  shared 
data  should  materially  contribute  to  the  solution 
of  this  critical  problem. 

FUTURE  TRENDS 

The  advance  of  large-scale  integrated  micro 
electronics  is  causing  a  revolution  In  computer 
industry.  Researchers  predict  that  it  will  be  possi¬ 
ble  to  fabricate  10°  gates  on  a  single  chip.  In¬ 
deed  some  very  large  scale  integration  (VLSI)  tech¬ 
nology  in  the  form  of  micro  computers,  micro  proces¬ 
sors,  and  related  system  device  elements  exists  at 
the  present  time.  Computer  system  approaches  using 
microprocessors  and  microcomputers  In  functionally 
distributed  architectures  are  not  far  over  the 
horizon. 

Functionally-modular  system  architectures  will 
drive  the  future  real-time  trainer  software  develop¬ 
ments  to  modular  software.  These  software  modules 
will  be  application-unique  and  will  be  dedicated  to 
individual  microcomputers  that  are  organized  and 
controlled  as  a  distributed  microcomputer  system. 

Such  a  system  will  not  be  the  same  as  the  conven¬ 
tional  master/slave  system  approach.  As  software 
modules  are  developed  for  various  classes  of  train¬ 
ers,  the  source  code  of  these-modules  (e.g.,  in 
ADA,  Pascal,  FORTRAN,  assembly  code)  can  be  stan¬ 
dardized  and  transported  between  trainers  of  the 
same  class.  This  will  be  a  significant  step  toward 
reducing  the  life  cycle  costs  of  trainers. 

In  the  past,  flight  trainers  have  generally 
been  designed  without  regard  to  the  instructional 
environment  of  which  they  will  become  a  part.  Sim¬ 
ply  stated,  a  flight  trainer  is  developed  as  a 
hardware  item.  Curricula  development  usually  was 
undertaken  by  the  using  activity.  The  task  of 
operating  the  device  and  developing  training  scenarios 
was  within  the  capability  of  the  assigned  Instructor 
and  operator  personnel .  With  the  complex  aircraft 
and  the  instructional  programs  required  to  support 
today's  training  needs,  it  has  become  obvious  that 
substantial  amounts  of  assistance  must  be  provided 
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Figure  6.  Flight  Trainer  Computing  System  Trends 
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to  Instructors  and  operators. 

A  6.4  Engineering  Development  program  entitled 
"Aviation  Weapons  System  Simulation"  was  Initiated 
in  1978.  The  objective  of  this  program  was  to 
conduct  research  so  that  instructor  and  operator 
workload  could  be  reduced  while  improving  the  ef¬ 
fectiveness  of  training  at  the  same  time.  The 
project  called  for  the  development  of  a  training 
system  that  was  individualized  and  adaptive  with 
voice  synthesis  and  performance  measurement  subsys¬ 
tems.  Such  an  instruction  system  strap-on  package 
is  about  to  be  delivered  for  integration  with 
Device  2F95A  F14  OFT.  This  modular  package  itself 
contains  a  computer  system  made  up  of  two  digital 
computers,  one-S/130  and  one-S/250  with  a  96  mega¬ 
byte  disc  for  mass  data  storage.  The  major  fea¬ 
tures  of  this  system  Include  management  and  record¬ 
keeping,  data  on  student  performance,  briefings  on 
problem  setup  and  scenario  generation  for  the 
instructor,  performance  measurement,  problem  diag¬ 
nosis  with  feedback  for  students,  and  use  of  voice 
technology  to  assist  the  Instructor/operator  (17). 

Although  there  is  Increased  sophistication  and 
complexity  in  evolving  aircraft  weapon  systems,  the 
major  trend  for  future  flight  simulators  will  be 
the  inclusion  of  a  complete  instructional  system 
with  a  training  strategy  at  the  outset  of  the 
development  process.  The  F14  OFT  strap-on  package 
is  Intended  to  demonstrate  the  application  of  In¬ 
structional  technology  to  complex  trainers.  Based 
on  the  evaluation  of  this  system  we  can  anticipate 
substantial  emphasis  in  the  Instructional  system 
aspect  of  major  trainer  developments  in  the  future. 

The  software  problem  has  been  highlighted. 

It  is  easy  to  propose  procedural ly  oriented  solu¬ 
tions.  However,  it  Is  somewhat  more  difficult  to 
change  organizational  and  management  structures  to 
effect  the  desired  results.  To  bring  order  to  the 
software  development  process,  it  will  be  broken 
down  into  Its  essential  elements  and  geared  to 
high-volume  throughput.  These  elements  are  design, 
manufacturing,  and  test. 

The  design  phase  will  rely  on  the  increased 
use  of  standards,  and  a  formal  design  release  and 
control  system.  An  independent  software  develop¬ 
ment  or  manufacturing  facility  (for  coding,  com¬ 
piling,  editing,  testing  and  documenting,  etc.), 
which  has  the  necessary  software  production  and 
configuration  control  machine  tools,  will  be  esta¬ 
blished.  Such  facilities  would  Improve  the  effec¬ 
tiveness  and  Increase  the  throughput  of  the  soft¬ 
ware  development  process.  And  finally,  a  fully 
Independent  test  and  quality  assurance  program  will 
be  established.  Software/hardware  Integration 
testing  will  be  included  in  this  phase.  The  esta¬ 
blishment  of  independent  design,  manufacturing  and 
testing  functions  will  not  remove  the  programmer 
from  the  software  development  process.  It  will 
create  a  number  of  specifications  within  the  field. 
It  will  also  provide  visibility  of  software  develop¬ 
ment  for  control  and  accountability  purposes. 
Software  systems  will  become  another  component  of 
system  development,  not  an  end  Into  Itself. 

CONCLUSION 

The  evolution  of  real-time  digital  simulation 
for  flight  trainers  has  been  traced  from  the  Ini¬ 
tial  research  studies  conducted  by  the  University 
of  Pennsylvania  which  were  initiated  in  1949. 


From  that  baseline  computer  system  design,  the 
computer  and  programming  technology  related  to  this 
field  have  gone  through  a  number  of  evolutions. 

These  evolutions  have  been  geared  to  the  technologi¬ 
cal  advances  in  the  computer  field.  Kith  respect 
to  computer  architecture,  the  trend  was  one  of 
going  from  expensive  CPU's  with  speed  and  memory 
capability  that  were  barely  adequate  for  the  task, 
to  the  present  system  architecture  of  inexpensive 
and  powerful  distributed  processors  based  on  a  func¬ 
tional  allocation  to  the  subsystems  Involved  in  the 
flight  trainers.  In  the  future,  highly  efficient 
distributed  processor  systems  can  reasonably  be 
expected.  Current  Iteration  requirements,  30  Hz, 
are  geared  to  synchronous-type  operation  with 
television-type  systems.  Iteration  rates  will  be 
Influenced  by  aircraft  on-board  computers  whose 
iteration  rates  may  exceed  the  30  Hz  standard  which 
has  been  established.  Progress  will  be  made  in  the 
use  of  FORTRAN  as  more  efficient  compilers  are 
developed  by  computer  manufacturers  for  real-time 
simulation.  The  magnitude  of  the  software  develop¬ 
ment  effort  requires  a  highly  disciplined  approach 
to  its  design,  manufacture,  quality  assurance,  and 
test  so  that  the  software  subsystem  can  take  Its 
place  as  a  mature  engineering  discipline.  ADA  will 
be  Introduced  as  soon  as  Its  development  Is  com¬ 
plete.  The  next  generation  of  flight  training  sys¬ 
tems  will  incorporate  intensive  use  of  instructional 
features  to  Improve  training  effectiveness  while 
minimizing  the  use  of  energy  and  hunan  resources. 
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DATA  ACQUISITION  AND  ANALYSIS  SYSTEM  AS  A  TRAINING  DEVICE  FOR 
SIMULATED  CONVENTIONAL  WEAPON  DELIVERY 

by 

C.F.G.M.  Hofman 

National  Aerospace  Laboratory,  NLR 
Amsterdam 
The  Netherlands 


1 .  INTRODUCTION 

The  tasks  to  be  performed  by  the  pilot  during 
conventional  weapon  delivery  in  high  speed  fighter 
aircraft  depend  on  the  systems  available  aboard  the 
aircraft,  and  on  the  type  of  weapon  used. 

Presently  the  main  part  of  the  air-to-ground  trai¬ 
ning  method  in  the  Netherlands  consists  of  the 
delivery  of  training  bombs  at  training  ranges. 

The  disadvantages  of  this  method  are: 

-  the  site  of  weapon  delivery  (and  in  many  cases 
also  the  direction  from  whicn  the  attack  is 
initiated)  is  often  the  same  (learning  effect!) 

-  the  performance  of  the  pilot  is  expressed  in  one 
quantity  only:  the  score.  The  release  conditions 
are  not  measured 

-  the  high  consumption  cf  weapons  (expensive!). 

In  order  to  avoid  these  disadvantages,  a  simple 
system,  called:  "Delivery  and  Impact  Analysis 
System"  (DIAS)  has  been  developed  and  tested  by  the 
National  Aerospace  Laboratory  NLR,  under  contract 
for  the  Royal  Netherlands  Air  Force  (RNLAF) . 

This  system,  based  on  a  photograraraetric  method, 
yields  release  conditions,  the  nominal  weapon  im¬ 
pact  position  and  the  weapon  time  of  flight.  Simu¬ 
lated  attacks  on  a  great  variety  of  realistic  tar¬ 
gets  can  easily  be  evaluated  and  validated  as  there 
is  no  need  to  drop  training  weapons.  Furthermore 
no  ground-based  instrumentation  in  the  target  area 
is  needed.  The  system  consisting  of  an  airborne 
data  acquisition  system  installed  in  the  aircraft 
and  a  ground-based  processing  and  analysis  system 
at  the  airbase  allows  a  debriefing  of  the  pilot 
within  half  an  hour  after  completion  of  the  mission. 
This  paper  gives  a  description  of  the  system. 
Attention  is  paid  to  the  system  requirements,  the 
system  evaluation  and  the  implementation  in  an 
operational  NF-5  squadron  of  the  RNLAF. 


2.  SYSTEM  REQUIREMENTS 

The  RNLAF  using,  among  others,  NF-5  aircraft 
to  carry  out  low  level  attacks  wished  to  have  a 
system  realized  for  that  type  of  aircraft  which 
should  meet  the  following  requirements: 

1 .  Accurate  assessment  of  pilot  performance  con¬ 
cerning  simulated  weapon  delivery  in  an  (semi) 
operational  environment  by  yielding: 

-  weapon  release  conditions  and 

-  weapon  impact  positions  for  each  attack  run 

-  statistical  overviews  of  the  mission  results 
in  relation  to  pilot,  weapon,  target,  dive 
angle  etc. 

2.  Training  of  pilots  concerning  weapon  delivery 
by  a  fast  feed-back  of  the  mission  results. 

3.  A  simple  system  characterized  by: 

-  a  minimum  of  equipment 

-  a  minimum  of  personel 

-  low  costs 


-  no  influence  on  pilot  procedures  during 
weapon  delivery 

-  easy  handling  of  the  airborne  system  on  the 
ground 

-  user's  friendly  processing  and  analysis 
system. 

1 .  No  ground  equipment  in  the  target  area. 

5.  No  aircraft  modifications. 

6.  No  influence  on  the  operational  status  and 
handling  qualities  of  the  aircraft. 

Although  the  system  has  no  war  task  and  therefore, 
no  back-up  system  is  needed,  the  RNLAF  expressed 
the  following  requirements  concerning  the  "mean  time 
between  failures"  (MTBF) : 

-  airborne  system:  100  flying  hours 

-  ground-based  system:  1000  working  hours. 

Based  on  the  number  of  required  "DIAS"  sorties  per 
aircraft  and  the  related  working  hours  of  the 
ground-based  system  one  failure  every  half  a  year 
may  occur. 


3.  SYSTEM  DESCRIPTION 

3 . 1  Introduction 

DIAS  consists  of  two  parts,  namely: 

-  an  airborne  data  acquisition  system  attached  to 
the  aircraft  and 

-  a  ground-based  data  processing  and  analysis  system 
at  the  airbase. 

The  airborne  system  contains  a  photo  camera  by  which 
at  the  moment  of  weapon  delivery  two  photographs 
of  a  once  measured  target  area  are  taken.  The  infor¬ 
mation  available  from  both  photographs  is  processed 
and  analyzed  with  the  aid  of  the  ground-based  system 
and  yields  the  results  of  the  attack  carried  out. 

The  ground-based  system  comprises,  among  others, 
a  minicomputer,  a  display  terminal  and  a  photo 
reader. 

A  "DIAS  squadron"  has  at  its  disposal  several  air¬ 
craft  equipped  with  an  airborne  system  and  one 
ground-based  system. 

3.2  Airborne  data  acquisition  system 

The  airborne  system  of  DIAS  contains  the 
following  components: 

-  a  photo  camera  (2k  x  36  mm),  ROBOT  motor  recorder 
36  C,  2b  VDC  with  an  exposure  time  of  1/500  s. 

The  camera  is  equipped  with  an  object-glass 
"Schneider  Xenon"  f/1.9  -50  mm  and  a  central  type 
shutter 

-  a  release  magneto  (solenoid) 

-  an  automatic  diaphragm  control  unit  of  which  the 
electronics  were  modified  and  integrated  with 

-  an  electronic  unit,  designed  to  control  the 
camera 

-  a  housing  provided  with  a  front  glass  and  a 
retractable  screen  controlled  by  an  electric  motor 
(TRW  Globe  Motors,  type  5  A  5^*8-8). 
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The  camera  is  attached  to  the  aircraft  in  an  up¬ 
right  position  with  the  optical  axis  pointing  for¬ 
ward,  11  degrees  nose  down  with  respect  to  the  air¬ 
craft's  longitudinal  axis.  No  relevant  part  of  the 
camera  field  of  view  is  obscured  by  any  store 
attached  to  the  centerline  pylon. 

At  the  moment  of  weapon  delivery  the  camera  is 
activated  by  pressing  the  bomb  button  (pilot's 
action)  while  for  the  second  photograph  the  camera 
is  activated  automatically  3h0  ms  later  by  the 
electronic  unit  which  also  controls  the  camera 
diaphragm  and  film  transportation. 

The  housing  of  the  airborne  system  has  been 
attached  to  the  aircraft  on  the  spot  of  the  nose 
part  of  the  centerline  pylon  which  has  been  removed. 
The  difference  between  the  original  and  "DIAS" 
centerline  pylon  of  the  NF-5A  concerning  the  exter¬ 
nal  shape  is  shown  in  figure  1 . 


The  housing  consists  of  two  parts  of  which  one  is 
attached  to  the  centerline  pylon  permanently  and 
contains  the  electronic  unit.  The  second  part,  the 
"DIAS  nose",  is  attached  to  the  first  part  by  two 
"cpiick  release  latches"  (easy  removable)  and  con¬ 
tains  the  photo  camera  (Fig. 2). 


Fig.  1  The  original-  and  DIAS  centerline  pylon  of  the 
NF-5A  aircraft 


Fig.  2  The  DIAS  nose  removed  from  the  modified 
centerline  pylon 
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After  completion  of  the  flight  the  "DIAS  nose",  is 
brought  to  the  squadron  building  where  the  film  is 
removed  (easy  accessible)  and  developed. 

If  no  DIAS  mission  is  carried  out  the  real  "BIAS 
nose”  is  replaced  by  a  ’'dummy  nose"  with  an  identi¬ 
cal  external  shape  (Fig. 3). 


The  housing  of  the  airborne  system  is  designed 
such  that  all  centerline  stores  can  be  carried  as 
before  (Fig. if). 


Fig.  3  The  real  and  dummy  DIAS  nose 


Fig.  if  The  rocket  bomb  dispenser  attached  to  the  DIAS 
centerline  pylon 


The  retractable  screen  which  protects  the  front 
glass  of  the  housing  against  dust  and  salt  (low 
level  missions)  will  be  opened  during  the  weapon 
delivery  phase  through  the  activation  (pilot’s 
action)  of  either  the  "bomb  arming  switch" 

(bombing)  or  the  "armament  position  selector  switch 
centerline"  (rocketry).  Pilot's  actions  during  a 
"DIAS  mission"  do  not  differ  from  those  to  be 
carried  out  normally  during  an  attack  mission. 

In  addition  to  the  modification  of  the  centerline 
pylon  construction  a  minor  modification  concerning 
electrical  wiring  inside  the  pylon  was  needed 
(power  supply,  signals  to  camera).  Aircraft  modifi¬ 
cations  are  not  involved. 

3,3  Ground-based  data  processing  and  analysis 
system 

The  ground-based  data  processing  and  analysis 
system  that  is  situated  at  the  airbase  consists  of 
the  following  components: 

-  PCD  photo  reader  ZAE  71. 

-  DEC  general  purpose  computer  PDP  11/3**  (1 6  bits, 
64  k  words  MOS  memory). 

-  DEC  LA- 36  printer  . 

-  Lear  Siegler  ADM-U2  visual  display  unit. 

-  2  RL01  disk  units  (5  M  bytes). 


(easy  accessible  and  short  time  for  development). 

-  Interactive  programming  facilities 
(editing  activities). 

-  System  reliability 

(mean  time  between  failures  is  1000  hours). 

-  Standardization  in  the  RNLAF. 

-  Costs . 

With  tue  aid  of  the  photo-reader  the  developed  film 
negatives  are  projected  on  a  high  resolution 
screen.  The  once  measured  reference  points  (terrain 
features)  in  the  target  area,  visible  on  the  pro¬ 
jection,  are  digitized  and  sent  to  the  computer. 

On  the  basis  of  this  information  together  with 
briefing  data  (meteorological  condit'ons,  aircraft 
weight  and  configuration  etc.  given  by  the  pilot) 
and  data  stored  on  the  magnetic  disk  (specifica¬ 
tions  of  camera,  target,  etc.)  the  analysis  of  the 
attack  is  carried  out. 

The  results  of  the  attack  run  are  displayed  on  a 
printer  output  (Tab.l). 

All  run  results  are  stored  on  disk  to  provide  a 
statistical  overview  of  the  results  in  relation  to 
pilot,  weapon,  target  etc.  (Tab.2). 


The  selection  of  the  components  of  the  ground-based 
system  has  been  based  on,  among  others,  the  follow¬ 
ing  requirements: 

-  Floating  point  hardware 

(accuracy  and  speed  of  calculation  program). 

-  High  order  language  with  real-time  facilities 
(Basic  2+) 

TABLE  1 


DIAS  RUN  RESULTS 


SORTIE: 

PILOT  : 

AIRCRAFT:  K- 
CAMERA: 

WEAPON: 

WIND:  (deg)/ 


RUN: 

FLAPS: 

DELAY  TIME: 
(kts) 


DATE: 

FLIGHT  POSITION: 

SCL:  FUEL:  (lbs) 

TARGET: 

(s)  SIGHT  SETTING  (mils) 

CODE:  QNH:  (mb)  OAT:  (°C) 


DELIVERY  PARAMETERS: 
TAS  (kts) 

IAS  (kts) 

DIVE  ANGLE  (deg) 
HEIGHT  (ft) 

SLANT  RANGE  (ft) 
NORMAL  LOAD  (g) 

PITCH  ANGLE  (deg): 
HEADING  (deg): 

PIPPER  POSITION: 

ON  PHOTOGRAPH 


PLANNED  ACTUAL 


BANK  ANGLE  (deg): 
TRACK  (deg): 
LONGITUDINAL  (mils): 
LONGITUDINAL  (55): 


ERROR 


(?) 

ANGLE  OF  ATTACK  (mils): 

LATERAL  (mils): 

LATERAL  (?): 


SIMULATED  IMPACT  POSITION:  RANGE  ERROR  (ft)  : 

DEFLECTION  ERROR  (ft)  : 

WEAPON  TIME  OF  FLIGHT  (s): 

TARGET  REF.  POINTS  NOT  ACCEPTED:  PHOTO  1:  PHOTO  2: 

AIRCRAFT  REF;  POINTS  NOT  ACCEPTED:  PHOTO  1:  PHOTO  2: 

(CONSIDERING  HEADING,  TRACK,  BANK  ANGLE  AND  CROSS  WIND  A  SIDE-SLIP  WAS  PROBABLY  INTRODUCED) 
(ACTUAL  DELIVERY  PARAMETERS  OUT  OF  BOMB  TABLE  RANGE) 
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TABLE  2 


STATISTICS  OF  DIAS  RUN  RESULTS 


DATE  OF  PRINTOUT:  OPERATOR:  VERSION : 

SELECTED  PARAMETER  VALUE  LOWER  BO'JND  UPPER  BOUND 

DATE 

SORTIE  NUMBER 
PILOT  CODE 
FLIGHT  POSITION 
AIRCRAFT  CODE 
CAMERA  CODE 
TARGET  CODE 
WEAPON  CODE 
PLANNED  TAS  (kts) 

PLANNED  DIVE  ANGLE  (deg) 

PLANNED  HEIGHT  (ft) 

LIFE  DELIVERY  (NO/YES) 


DELIVERY  PARAMETERS  (NUMBER  OF  RUNS  =  ) 

TAS  ERROR  (kts) 

DIVE  ANGLE  ERROR  (deg) 

BANK  ANGLE  (deg) 

NORMAL  LOAD  (g) 

HEIGHT  ERROR  (%) 

SIMULATED  IMPACT  (NUMBER  OF  RUNS=  ! 

RANGE  ERROR  (ft) 

DELFLECTION  ERROR  (ft) 

ACTUAL  IMPACT  (NUMBER  OF  RUNS=  ) 

RANGE  ERROR  (ft) 

DEFLECTION  ERROR  (ft) 

DUD  (NUMBER  OF  RUNS=  ) 

DUD  -  RATE  =  % 


h.  SYSTEM  EVALUATION 

L ■ 1  General 

After  having  carried  out  an  investigation 
about  the  feasibility  of  the  photogrammetric  method 
to  be  applied  for  DIAS  (l),  the  analysis  technique 
was  tested.  For  the  system  performance  tests,  life 
deliveries  on  a  weapon  range  were  executed  (2). 

Use  was  made  of  an  instrumented  NF-bA  test  aircraft 
(3)  equipped  with  a  DIAS  airborne  system  (proto¬ 
type  ) . 

The  processing  of  data  (photographs)  and  the  data 
analysis  were  carried  out  with  the  aid  of  photo¬ 
reader  and  computer  facilities  of  the  NLR. 

In  order  to  establish  the  airworthiness  of  the 
airborne  system,  tests  on  the  ground  as  well  as 
during  flight  were  carried  out. 

h.g  System  performance 

U . 2 . 1  General 

Before  DIAS  was  realized  an  investigation  was 
carried  out  to  establish  the  feasibility  of  such 
a  system.  During  this  study  the  analysis  technique, 
based  on  a  photogrammetric  method  was  developed. 
Attention  is  paid  to  the  analysis  method  applied  in 
DIAS  and  information  is  given  about  the  accuracy 
of  the  system.  Finally  the  performance  flight  tests 
carried  out,  to  verify  the  analysis  technique  are 
described. 


MEAN  ST. DEVIATION 


h.2.2  Calculation  technique 

As  already  mentioned  befor--  mission  planning  data, 
available  through  the  pilot,  constant  system  data, 
stored  in  the  data  base  of  the  ground-based  system 
and  the  information  available  from  two  photographs, 
tak  n  of  the  target  area  at  the  moment  of  simulated 
weapon  delivery  are  needed  to  determine  the  attack 
performance. 

Table  3  gives  an  overview  of  these  data. 

In  the  following  the  analysis  method  is  described  in 
general  terms. 

The  photogrammetric  method. 

The  photogrammetric  method,  used  to  determine  the 
position  and  attitude  of  a  camera  with  the  aid  of  a 
photograph,  taken  of  a  defined  object,  is  based  on 
tile  fact  tha‘  there  is  only  one  position  from  which 
this  ibject  con  be  seen  the  way  it  is  displayed  on 
tie'  pnotograph . 

Expressed  in  mathematical  terms: 

the  camera  has  to  be  translated  as  well  as  rotated 
such  that  the  image  vectors  on  the  photograph,  trans¬ 
formed  to  the  object  co-ordinate  system,  cover  com¬ 
pletely  : he  object  vectors  in  length  and  direction. 
More  background  information  about  the  mathematical 
procedures  is  given  in  (L),(b)  and  (6). 

Camera  attitude  with  respect  to  the  aircraft. 

The  bottom  of  the  forward  part  of  the  aircraft 
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TABLE  3 

Data  needed  for  the  analysis  of  a  DIAS  mission 


mission 

subject  planning  data  base  mission  data 

_ data _ 

aircraft  -  tail  number  -  position  of 

-  configuration  reference  points 

-  fuel  remaining  and  camera 

over  target  -  configuration 

data 


-  gun  sight 
position 


target  -  target 

number 


-  position  of 
reference  points 
including  target 

-  target  altitude 
above  mean 

sea  level 


camera  -  camera  -  focus  -  2  photos 

number  _  measures 

of  negative 

-  time  period 
between  photos 

weapon  -  weapon  type  -  ballistic 

-  fuse  arming  table 

delay  time  _  maximum  fuse 

setting  arming  delay 

time  setting 

-  positive  tolerance 
fuse  arming  delay 
time  setting 

meteorological*  -  wind 

conditions  in  -  temperature 

target  area  -  pressure  (QNH) 

planned  -  dive  angle** 

delivery  -  airspeed** 

data  -  release  altitude** 

-  sight  depression 
_  of  gun  sight 

*  Normally  forecast  data  are  available. 

In  case  of  life  deliveries  on  a  weapon  range  actual  data  are  taken. 

In  case  the  pilot  experiences  other  wind  conditions  than  forecasted,  and  he  adjusts  his 
planned  delivery  parameters,  the  changed  wind  conditions  will  be  taken  for  the  analysis. 
**  These  data  are  not  needed  for  the  analysis  but  are  used  to  indicate  pilot's  errors. 


fuselage  has  been  provided  with  U  (aircraft)  refe¬ 
rence  points  of  which  the  positions  together  with 
the  camera  position  have  been  measured  once  in 
relation  to  the  aircraft  co-ordinate  system.  These 
reference  points,  visible  on  the  photograph,  are 
used  to  calibrate  the  camera  attitude  for  each 
attack  run  with  the  photogrammetric  method. 

Position  and  attitude  of  the  aircraft  with  respect 
to  the  target. 

Every  DIAS  target  is  selected  by  the  requirement  of 
having  terrain  features  which  can  serve  as  target 
reference  points.  The  position  of  each  point  is 
measured  once  (infrared  theodolite)  in  relation  to 
the  target.  With  these  reference  points,  visible  on 
the  photograph,  the  position  and  attitude  of  the 
camera  and  thus  of  the  aircraft  can  be  determined. 
Having  analyzed  the  photograph,  taken  at  the  moment 
of  weapon  delivery,  the  following  release  conditions 
are  already  known: 


-  release  altitude 

-  heading  of  aircraft. 

Together  with  the  second  photograph,  taken  3^0  ms 
after  weapon  delivery,  the  following  parameters  are 
determined: 

-  ground  speed 

-  ground  track. 

Angel  of  attack,  dive  angle  and  true  airspeed. 

Because  of  the  fact  that  the  dive  angle  is  one  of 
the  most  critical  delivery  parameters  much  attention 
is  paid  to  calculate  this  variable  as  accurate  as 
possible. 

During  the  performance  flight  tests  it  appeared  that 
the  calculation  of  the  dive  angle  by  using  both 
known  positions,  was  too  inaccurate  because  very 
often  a  curved  flight  path  was  flown. 

For  that  reason  the  dive  angle  is  calculated  as  a 
function  of  the  angle  of  attack,  true  airspeed 
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(function  of  ground  speed  and  dive  angle),  meteo¬ 
rological  and  aircraft  data  and  taking  into  account 
pitch  rate  effects. 

The  dive  angle  calculation  is  executed,  using  an 
iterative  process  because  both  angle  of  attack  and 
dive  angle  are  unknown. 

Note:  If  the  wind  information  is  incorrect  the  true 
airspedd  will  also  be  incorrect.  But,  as  the 
pilot  plans  (and  executes)  the  attack  with 
the  wind  information  available,  the  results 
are  also  based  on  this  information. 

The  normal  load  factor. 

The  normal  load  factor  is  derived  from  the  true 
airspeed  and  angle  of  attack  taking  into  account 
pitch  rate  effects. 

Side-slip  indication. 

In  case  the  aircraft  ground  track  and  heading  are 
not  identical,  the  difference  may  be  caused  by 
either  drift  or  slip  or  by  both. 

If  the  difference  does  not  match  with  the  cross 
wind  component  and  aircraft  speed  it  is  concluded 
that  the  pilot  initiated  a  side-slip  condition. 

Tnis  is  indicated  in  the  run  results. 

Pipper  position. 

The  pipper  position  in  the  ground-based  reference 
frame,  at  the  moment  of  weapon  delivery  is  calcula¬ 
ted  by  using  the  position  of  the  gun  sight  with 
respect  to  the  aircraft  reference  frame,  the  air¬ 
craft  position  in  relation  to  the  target  and  taking 
into  account  the  angle  of  attack  and  the  sight 
depression  set  by  the  pilot. 

Weapon  impact  position  and  weapon  time  of  flight. 

For  the  calculation  of  the  (nominal)  impact  posi¬ 
tion  and  time  of  flight,  use  is  made  of  ballistic 
weapon  tables,  also  used  by  the  pilots  to  plan  the 
delivery. 

In  case  the  actual  value  of  a  delivery  parameter 
is  not  available  in  the  table,  an  interpolation  is 
performed,  using  a  second  degree  polynomial. 

k.2.3  Accuracy 

In  this  section  the  most  important  error  sources 
of  the  system  are  described.  Furthermore  an  experi¬ 
ment  carried  out,  to  determine  the  errors  and  their 
influence  on  the  results  are  discussed.  Finally  the 
overall  accuracy  will  be  given. 

The  most  important  error  sources  are: 

-  measurement  of  target-  and  aircraft  reference 
points 

-  photo-camera 

-  read-out  of  photographs 

-  delay  time  of  weapon  release. 

Because  the  system  is  used  in  combination  with  a 
flying  aircraft  some  system  errors  are  caused  by 
motion  only.  For  that  reason  distinction  is  made 
between  "dynamic"  and  "static"  errors. 

"DYNAMIC"  ERRORS 

Shutter  time  of  camera. 

The  nominal  shutter  time  of  the  cameras  in  use  is 
2  m3  which  is  tested  for  each  camera.  This  value 
appears  to  be  correct. 


During  the  exposure  of  the  film  the  aircraft  covers 
a  certain  distance  in  longitudinal  direction. 
Furthermore  the  left  hand  side  of  the  photograph  is 
exposed  1  ms  earlier  than  the  right-hand  side 
causing  a  position  shift  of  reference  points  on  the 
left-hand  side  with  respect  to  the  points  on  the 
right-hand  side. 

Taking  into  account  an  aircraft  speed  of  500  kt  and 
a  slant  range  of  2000  ft  to  the  target,  this  shutter 
time  causes  a  position  error  of  about  3  ft  and  an 
attitude  error  of  about  0.02°. 

Elapsed  time  between  both  photographs. 

The  electronics  which  control  the  time  period 
between  both  photographs  is  adjusted  to  3^0  ms. 

Tests  on  the  ground  as  well  as  in  flight  showed  that 
the  maximum  deviation  is  1  ms. 

Based  on  an  aircraft  speed  of  500  kt  and  a  timing 
error  of  1ms  a  position  error  of  about  1  ft  is  caused. 

Delay  time  of  weapon  release. 

With  flight  tests  during  which  actual  deliveries 
were  carried  out  the  delay  times  between  "pickle" 
moment  (pilot's  action)  on  the  one  hand  and  the 
real-  and  "DIAS"  release  moment  (first  photograph) 
on  the  other  hand  were  measured.  The  "DIAS"  release 
occurred  33  ms  after  "pickle"  while  the  actual 
release  took  place  25  to  kO  ms  after  "pickle". 

Taking  into  account  both  the  camera-  and  actual 
weapon  release  time  delay  a  difference  of  10  ms 
exists. 

Based  on  that  difference  and  an  aircraft  speed  of 
500  kt  an  impact  error  of  about  8  ft  is  caused. 

"STATIC"  ERRORS 

Measurement  of  target-  and  aircraft  reference  points. 

The  positions  of  the  target  reference  points  (geo¬ 
graphical  features)  with  respect  to  the  target  are 
measured  with  the  aid  of  an  infrared  theodolite. 

The  accuracy  of  the  measurements  is  within  10  cm 
for  the  relevant  distances. 

The  positions  of  the  aircraft  reference  points  with 
respect  to  the  photo-camera  are  measured  with  the 
aid  of  a  theodolite.  The  measurements  are  carried 
out  with  the  aircraft  levelled  (also  used  for  cali¬ 
bration  of  the  gun  sight).  The  accuracy  of  the  mea¬ 
surements  is  within  1  mm. 

Camera  focus . 

The  focal  distance  (50  mm  nominal)  of  each  camera  in 
use  is  calibrated  with  an  accuracy  of  o.l  %. 

Lens  distortion. 

To  avoid  the  influence  of  lens  distortion  (edges  of 
the  lenses)  reference  points  visible  on  the  edge  of 
the  photograph  (within  10  %  of  the  width)  are  not 
used  for  the  analysis. 

Read-out  of  photographs. 

The  photographs  (negative)  is  enlarged  1  times  on 
the  photo-reader  screen  which  means  that  the  pro¬ 
jection  measures  are  about  3^  x  50  cm. 

Read-out  tests  showed  that  well  defined  reference 
points,  visible  on  the  photograph  can  be  read  out 
with  an  accuracy  of  0.2  mm. 

The  linearity  of  the  photo-reader  which  has  been 
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tested,  is  excellently. 

Experiment . 

In  order  to  establish  the  total  "static"  error  an 
experiment  was  carried  out  during  which  with  a 
calibrated  camera  (fixed  position)  photographs  were 
taken  of  an  area  with  8  reference  points.  The 
greatest  distance  between  the  camera  and  the  refe¬ 
rence  points  was  about  2000  ft.  With  the  aid  of 
the  DIAS  calculation  process  the  camera  position 
and  attitude  were  determined.  This  experiment 
showed  a  maximum  position  error  of  4.5  ft  (standard 
deviation  1.5  ft)  and  maximum  attitude  errors  of 
less  than  0.1°  (standard  deviation  0.03°)* 

OVERALL  ACCURACY 

Taking  into  account  the  experimental  "static" 
errors  as  well  as  the  "dynamic"  errors  (except  for 
"delay  time  of  weapon  release")  it  is  concluded 
that  the  overall  accuracy  of  the  DIAS  position  and 
attitude  determination  process  is  given  by  8  ft  and 
0.1°  (95  %  confidence  level)  respectively. 

Based  on  the  position  error  and  the  time  period  of 
340  ms  between  both  photographs  the  maximum  air¬ 
speed  error  (95  %  probability)  is  10  kt. 

Making  use  of  the  ballistic  tables  available  and 
taking  into  account: 

-  the  position-,  attitude-  and  airspeed  errors 
mentioned  before 

-  a  less  accurate  calculation  of  the  angle  of 
attack  and  dive  angle  (accuracy  is  0.15°) 
and  furthermore  the  variation  of: 

-  weapon  characteristics  (weight,  drag) 

-  aircraft  characteristics  (e.g.  lift  curve  slope) 

-  meteorological  data 

-  delay  time  between  "DIAS"  and  actual  release 
mome;.‘ 

The  following  impact  errors*  (circular  error  pro¬ 
bable;  CEP)  were  expected  during  the  performance 
flight  tests  to  be  carried  out: 

-  bombing  CEP  =  50  ft 

-  gunnery  CEP  *  55  ft 

-  rocketry  CEP  =  oO  ft. 

4.2,4  Performance  flight  tests 

In  order  to  evaluate  the  DIAS  performance ,  flight 
tests  were  carried  out  with  an  instrumented  NK-5A 
test  aircraft  of  the  RNLAF  («?), 

During  the  program  15  missions,  consisting  of  in 
total  53  attack  runs  were  carried  out  on  a  weapon 
range.  Delever Les  were  made  with  20  MK-IOo's,  28 
BDU- 3 5B r s  and  5  MK-82  R’s. 

A  number  of  8  BDU-33B*s  were  delivered  under  so- 
called  "bunting  relase"  conditions  (large  deviations 
from  "steady  state"  conditions,  especially  side-slip). 

In  the  following,  information  needed  for  a  good 
understanding  of  the  flight  tests  results  obtained, 
is  given: 

1.  The  target  area  was  provided  wigh  9  reference 
points  equally  distributed  over  the  area.  All 
points  were  (always  visible)  taken  into  account 
for  the  analysis. 

The  actual  impact  positions  were  measured 
accurate ly . 

3.  Four  photographs  within  1  s  were  taken,  to 
determine  the  best  time  interval  for  future 
DIAS  photographs. 

•difference  between  the  actual  and  predicted  impact 
positions . 


4,  The  release  conditions  varied  as  follows: 


dive  angle 
slant  range 
release  height 
airspeed 
normal  load 


0  -15  deg 
1200  -  2300  ft 
170  -  630  ft 
415  -  500  kt 
0.9  -  1  .1*  g. 


5.  Use  was  made  of  actual  meteorological  data. 

From  the  performance  flight  tests  the  following 
conclusions  can  be  summarized: 

1.  The  circular  error  probable  (CEP)  and  standard 
deviation^ )  of  the  differences  between  the 
actual  and  predicted  (DIAS)  impact  positions, 
were: 

a.  BDU-33B 

20  deliveries  "steady  state"  release  condi¬ 
tions  : 

CEP  =  3r’  ft  S  =  32  ft 
8  deliveries  "bunting"  release  conditions: 
CEP  =  81  ft  S  =  1*7  ft 

b.  MK-106 

20  deliveries  "steady  state"  release  condi¬ 
tions  : 

CEP  =  1*2  ft  S  =  21  ft 

c.  MK-82R 

5  deliveries  "steady  state"  release  condi¬ 
tions: 

CEP  =  1*3  ft  S  =  3**  ft. 

2.  The  values  of  the  delivery  parameters  obtained 
by  the  aircraft  special  PCM  instrumentation 
system  and  those  obtained  with  the  DIAS  method 
agreed  within  the  measurement  accuracy  of  both 
systems . 

3.  During  the  flight  tests  it  was  found  that  two 
photographs  taken  with  a  time  interval  between 
300  and  350  ms  would  be  most  appropriate  to 
the  analysis. 

1*.  Although  DIAS  is  able  to  analyse  (by  approxima¬ 
tion)  non-steady  state  deliveries,  it  has  to 
be  emphasized  that  the  results  obtained  in 
those  cases  must  be  considered  with  some  reser- 


During  the  flight  tests  no  gunnery  and  rocketry 
attacks  were  carried  out.  It  is,  however,  expected 
that  the  impact  errors  for  both  attack  types  are 
within  the  accuracy  mentioned  in  the  previous 
s-ction. 

1*,3  Airworthiness  of  the  airborne  data  acquisition 
system 

Ground-based  tests. 

To  establish  the  airworthiness  of  the  airborne  sys¬ 
tem  the  following  tests  were  carried  out: 

-  altitude  test 

-  temperature/humidity  test 

-  vibration  test 

-  electro  magnetic  interference  (EMI)  test. 

An  EMI  test  with  the  system  attached  to  the  aircraft 
was  also  carried  out. 

Te  tests  were  executed  for  the  prototype  system  as 
well  as  for  the  series  production. 

In  summary  it  can  be  stated  that  the  airborne  system 
has  withstanded  the  tests  excellently. 

Flight  tests. 

As  is  customary  when  new  aircraft  configurations 
are  to  be  certified,  the  following  subjects  are 
investigated : 

-  structural  strength 

-  flying  qualities 


-  flutter 

-  aircraft  performance 

-  store  separation. 

Structural  strength. 

The  DIAS  centerline  pylon  nose  has  to  withstand 
the  loads,  that  will  be  imposed  under  the  flight 
condition  listed  in  table  4. 

TALBE  4 


acceleration  (p;' 

airspeed 

(kt) 

Mach 

number 

longitudinal 

lateral 

vertical 

i  2.5 

±  1.5 

-1.5/+7.2 

720 

1.7 

The  structural  strength  of  the  nose  is  considera¬ 
bly  in  excess  of  the  requirement,  as  stiffness, 
necessary  to  obtain  a  good  camera  platform,  was 
the  dominant  factor  in  determining  the  dimensions 
of  the  construction.  Hence,  it  was  decided  not  to 
evaluate  the  structural  strength  aspects  by  means 
of  flight  tests,  but  only  to  demonstrate  structu¬ 
ral  integrity,  by  flying  up  to  the  limits,  imposed 
by  the  aircraft  itself. 

Plying  qualities. 

The  location  of  the  DIAS  nose  is  such,  that  no 
changes  in  the  distribution  of  the  lift  over  the 
wing  will  occur,  nor  will  the  airflow  over  the 
elevators  be  affected.  Thus,  no  changes  in  the 
overal  flying  qualities  of  the  aircraft  due  to 
aerodynamic  effects  will  occur. 

The  mass,  added  due  to  DIAS  (30  lbs)  results  in  a 
shift  in  the  centre  of  gravity  of  the  aircraft, 
but  the  shift  will  be  small  (less  than  0.1  1 
m.a.c.).  Furthermore,  the  shift  will  be  in  a 
forward  direction,  which  is  beneficial  for  flying 
qualities . 

Based  on  these  considerations  it  was  decided  that 
a  special  in  flight  evaluation  of  flying  qualities 
was  not  necessary.  No  anomalies  were  noted  during 
the  flights  carried  out  so  far. 

Flutter. 

Past  experience  (extensive  analysis  of  various 
NF-5  configurations,  verified  by  flight  tests)  has 
shown  that  the  amount  of  mass,  installed  at  the 
centreline  pylon  hardly  affects  the  flutter 
behaviour  of  the  aircraft.  As  the  mass  addition 
due  to  DIAS  is  rather  low,  it  was  decided  that  no 
flutter  analysis  was  required. 

Aircraft  performance. 

To  establish  the  increase  in  drag  number,  perform¬ 
ance  measurements  were  carried  out  and  compared 
with  measurements  executed  during  flights  with  the 
normal  centerline  pylon. 

The  results  of  the  tests  showed  that  the  effect  of 
the  DIAS  nose  on  aircraft  performance  is  quanti¬ 
fied  by  an  increase  of  the  basic  configuration 
drag  number  (7)  with  5  counts  which  is  negligible. 

Store  separation. 

The  change  in  geometry  of  the  centerline  pylon 
will  affect  the  air  loads  acting  on  captive  stores. 
Thus  the  behaviour  of  these  stores  during  separa¬ 
tion  may  be  affected.  To  evaluate  the  magnitude  of 
the  change  measurements  were  carried  out  using  the 


air  load  measuring  store  described  in  (8).  Hie 
results  were  compared  with  data  obtained  earlier  in 
combination  with  a  normal  centerline  pylon. 

The  normal  and  lateral  force  components  as  well  as 
yawing  and  rolling  moment  coefficients  were  identi¬ 
cal  in  both  cases. 

With  the  normal  pylon,  no  effect  of  Mach  number  on 
the  pitching  moment  coefficient  was  present,  where¬ 
as  with  the  DIAS  nose  there  is.  Furthermore  with 
the  DIAS  nose  the  moment  coefficient  is  more  nose 
up  at  low  angles  of  attack  and  low  Mach  numbers . 

With  the  knowledge  of  the  separation  behaviour  of 
empty  centerline  tanks  (low  density)  which  is  most 
critical  (9)  and  because  of  the  measured  small 
increase  of  the  pitching  moment  coefficient  (0.011) 
caused  by  the  DIAS  nose  it  has  been  concluded  that 
the  effect  of  the  change  in  local  air  flow  due  to 
the  DIAS  nose  on  store  separation  is  negligible  so 
that  store  separation  flight  limits  are  not  affected. 

Subsequently  demonstration  flights,  covering  the 
program  of  the  airworthiness  test  flights  (excluding 
performance  items ) ,  a  "cold  soak" ,  followed  by 
flight  in  a  warm,  moist  atmosphere ,  while  operating 
the  DIAS  system,  were  carried  out.  A  proper  opera¬ 
tion  of  the  system  during  the  most  extreme  condi¬ 
tions  was  demonstrated. 


5.  IMPLEMENTATION  OF  DIAS  IN  AN  OPERATIONAL 
NF-5  SQUADRON 

Because  of  the  results  obtained  the  RNLAF 
decided  to  realize  DIAS  for  "operational"  use. 

For  the  present  one  NF-5  squadron  of  the  RNLAF  has 
been  equipped  with  DIAS. 

The  squadron  has  at  its  disposal  a  ground-based 
system  and  5  aircraft  equipped  with  an  airborne 
system. 

In  total  1000  DIAS  sorties,  consisting  of  2000  to 
4000  attack  runs,  will  be  carried  out  per  year.  A 
large  number  of  targets  are  available. 

Dependent  on  the  results  to  be  obtained  during  the 
first  half  year,  more  aircraft  will  be  involved  and 
other  NF-5  squadrons  will  be  equipped  with  DIAS. 

At  the  moment  the  squadron  is  not  yet  "DIAS  opera¬ 
tional",  so  no  more  information  herein  can  be  given. 
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ABSTRACT 

Programs  for  generating  simulated  visual  and  far-infrared  imagery  of  Soviet  combatant 
ships  were  implemented  on  a  computer  system  at  the  Naval  Training  Equipment  Center.  These 
programs  were  used  in  a  system  definition  study  to  determine  the  characteristics  that  a 
real-time  computer  image  generation  system  must  have  in  order  to  satisfy  trainer  performance 
requirements  for  processing  capacity  and  resolution.  The  implemented  system  consists  of 
data  bases  at  various  levels  of  detail,  visual  and  far-infrared  sensor  models,  and  image 
generation  capabilities  that  include  a  flexible  set  of  options  for  various  viewing  condi¬ 
tions  and  special  features. 


INTRODUCTION 

The  Naval  Training  Equipment  Center  (NTEC)  is 
developing  the  Advanced  Visual/Near-Visual  Subma¬ 
rine  Periscope/Electro-Optic  Infrared  Sensor  Simu¬ 
lation  (AVEOSS)  prototype  trainer.  As  part  of  the 
supporting  R&D  effort,  Technology  Service  Corpora¬ 
tion  (TSC)  implemented  general  computer  image 
generation  (CIG)  programs  as  part  of  a  system 
definition  study  to  determine  the  characteristics 
necessary  for  the  trainer  system.  The  scenario 
involves  the  Osa,  Kashin,  and  Kiev  Soviet  combat¬ 
ant  ships  viewed  through  both  visual  and  far- 
infrared  sensors. 

Because  the  most  significant  parameters  in  the 
definition  study  were  level  of  detail  and  resolu¬ 
tion  at  varying  ranges  and  aspects,  the  CIG  pro¬ 
grams  and  models  had  to  allow  the  user  complete 
operational  control.  To  this  end,  the  geometric 
ship  models  were  constructed  from  planar  surfaces, 
ranged  in  level  of  detail  from  500  to  4000  poten¬ 
tially  visible  edges,  and  could  be  viewed  from  any 
aspect  angle  and  range  with  display  formats  from 
512  to  2048  raster  lines  per  TV  frame. 

This  paper  describes  the  CIG  programs  and  data 
bases  implemented  at  NTEC  in  terms  of  geometric 
data  base  modeling,  visual  and  infrared  models,  and 
frame  generation  characteristics.  The  data  bases 
developed  use  modular,  hierarchical  construction  so 
that  complex  scenes  can  be  processed  and  sensor- 
ship  interaction  can  be  blended  with  the  hidden- 
surface  and  shading  requirements  for  image  genera¬ 
tion.  The  visual  and  infrared  sensor  algorithms 
include  reflectivity  models  and  more  complex  com¬ 
ponents  for  determining  the  radiant  emittance  of  a 
surface  in  the  far-infrared  spectrum.  The  sensor 
models  incorporate  the  effects  of  ship  surface 
materials  and  the  intervening  (user-selected) 
maritime  environment.  The  frame  generation  routine 
employs  a  list-directed  priority  algorithm  for 
determining  hidden  surfaces.  Included  along  with 
the  user-selected  parameters  for  level  of  detail, 
range,  aspect,  resolution,  and  magnification  are 
options  for  edge  smoothing  and  smooth-surface 
shading. 


*The  work  leading  to  this  paper  was  performed 
under  NTEC  Contract  N61339-79-C-0151. 


The  CIG  methodology  used  in  the  system  is 
described  next,  followed  by  sample  imagery  of  the 
Kashin  and  Kiev  models  taken  from  a  high-resolution 
Di  corned  image  recorder.  Recommendations  for 
further  efforts  in  data  base  modeling  are  given  in 
the  last  section. 


CIG  METHODOLOGY 

The  system  implemented  at  NTEC  (see  block 
diagram  in  Figure  1)  generates  shaded  computer 
imagery  in  three  distinct  steps.  First,  a  scene 
(e.g.,  a  ship  on  the  ocean)  and  the  objects  com¬ 
posing  it  are  stored  in  a  geometric  data  base  in  a 
meaningful  and  useful  format.  Second,  the  tone  (or 
gray  level)  of  each  surface  is  predicted.  Third, 
the  surfaces  are  projected  onto  a  screen  and  a 
shade  assigned  to  each  pixel  for  the  surfaces  view¬ 
able  in  that  pixel. 


Figure  1.  General  CIG  System  and  Interface 

The  geometric  data  base  consists  of  planar 
polygonal  bounded  surfaces  defined  by  vertex 
points.  A  material  identification  code  Is  associ¬ 
ated  with  each  surface.  Surfaces  that  are  locally 
coherent  are  grouped  into  objects.  A  list-directed 
priority  scheme  employing  a  tree-structured  direc¬ 
tory  Is  used  in  organizing  the  data  base  In  order 
to  determine  hidden  surfaces.  The  data  base  con¬ 
verter  in  Figure  1  perfo.ms  three  primary  tasks: 


393 


1.  Convert  the  data  base  from  ASCII  to  a 
binary  format. 

2.  Set  up  a  tree-structured  directory  with 
pointers  to  the  surfaces. 

3.  Compute  the  outward  normal  of  each 
surface. 

Since  these  tasks  are  performed  early  in  the  pro¬ 
cess,  they  do  not  have  to  be  repeated  for  each 
frame  generated.  Thus,  while  subsequent  process¬ 
ing  is  repeated  as  conditions  change,  data  base 
conversion  is  performed  only  once  in  the  life  of  a 
data  base. 

The  toning  process  for  visible  and  infrared 
sensors  is  performed  independent  of  frame  genera¬ 
tion.  The  toning  routine  may  be  thought  of  as  a 
"black  box"  which  computes,  for  a  given  sensor  and 
set  of  environmental  conditions,  a  single  tone  for 
an  entire  surface  or  a  tone  for  each  vertex  of  a 
surface.  Material  descriptions  are  associated 
with  each  surface  to  provide  visible  reflectance, 
interior  conditions,  absorptivity,  emissivity, 
surface  orientation,  and  surface  roughness. 
Environmental  conditions  include  solar  and  sky 
loading,  air  and  water  temperature,  and  humidity. 

The  frame  generation  routine  is  central  to 
the  system.  In  general,  all  other  elements  of  the 
system  support  the  frame  generator.  With  a  toned 
data  base  and  viewing  parameters  as  input,  the 
frame  generator  outputs  a  synthetic  image  via  two 
subprocesses: 

1.  Geometric  processing,  in  which  the 
surfaces  of  the  data  base  are  ordered, 
translated,  and  rotated  relative  to  the 
viewer,  and  then  clipped  against  planes 
bounding  the  field  of  view. 

2.  Raster  processing,  in  which  each  raster 
is  generated  by  determining  the  view¬ 
able  surfaces  in  each  pixel  and  then 
assigning  the  appropriate  shade  for 
each  pixel. 

Geometric  Oata  Base  Modeling 

Basic  Geometric  Considerations.  The  basic 
primitive  used  to  build  the  geometric  model  is  a 
one-sided  polygonal  bounded  surface  that  can  be 
convex  or  concave.  It  is  defined  by  a  list  of 
vertex  points,  ordered  counter-clockwise  from  the 
viewable  side.  Since  each  surface  is  planar,  it 
possesses  a  unique  outward  normal.  Locally  coher¬ 
ent  surfaces  are  grouped  together  to  form  an 
object.  At  the  object  level ,  a  number  of  useful 
operations  may  be  performed,  most  notably  redefini¬ 
tion  or  specification  of  the  level  of  detail  and 
rapid  elimination  of  an  object's  surfaces  from  the 
field  of  view. 

Many  of  the  planar  surfaces  in  a  ship  data 
base  are  patchwork  approximations  to  curved  sur¬ 
faces.  To  perform  curved-surface  shading  and  make 
the  surfaces  appear  smooth,  the  surface  normal 
must  be  supplemented  by  defining  an  outward  normal 
for  each  vertex.  Although  in  some  cases,  such  as 
the  ship's  hull,  the  actual  contour  cannot  be 
determined,  selected  points  on  the  contour  can  be 
located.  In  such  a  case,  a  vertex  normal  can  be 
approximated  by  averaging  the  normals  of  surfaces 


that  share  the  common  vertex.  In  many  cases, 
simple  geometric  entities  such  as  spheres,  cylin¬ 
ders,  and  cones  (either  complete  or  truncated)  can 
be  originally  specified  for  objects.  The  objects 
can  then  be  transformed  into  a  patchwork  of  sur¬ 
faces. 

The  use  of  geometric  entities  which  can  be 
parametrically  specified  during  data  base  develop¬ 
ment  impacts  CIG  in  two  significant  ways:  1)  the 
contours  are  known  exactly  and,  consequently,  so 
are  the  vertex  normals;  2)  the  level  of  detail 
needed  to  represent  an  object  at  various  ranges  is 
easily  adjusted.  As  the  chosen  level  of  detail 
for  a  data  base  increases,  so  does  the  fineness  of 
the  patchwork  used  to  approximate  the  curved 
object.  For  example,  for  the  tubular  superstruc¬ 
ture  of  a  tower,  cylindrical  approximations  can  be 
used  for  high  detail  at  close  range,  whereas  only 
a  single  strip  is  needed  for  lower  levels  of 
detai 1 . 


Data  Base  Organization.  Local  surfaces 
listed  by  their  vertices  constitute  the  basic 
representation  of  an  object  in  a  geometric  data 
base.  However,  the  overall  design  of  a  geometric 
data  base  can  significantly  impact  CIG  system 
capabilities.  In  other  words,  while  the  basic 
geometric  data  are  necessary  for  generating  com¬ 
puter  imagery,  the  organization  of  the  data  base 
may  help  to  solve  problems  such  as  hidden-surface 
determination  as  well  as  allow  the  structured, 
modular  development  of  complex  scenes. 

The  hidden-surface  problem  encountered  in  the 
frame  generation  process  is  essentially  solved 
a  priori  by  using  a  list-directed  priority  scheme 
when  developing  the  geometric  data  base. 1,2  The 
drawbacks  of  this  solution  are  that  it  somewhat 
restricts  the  arrangement  of  surfaces  and  objects 
and  increases  the  cost  of  data  base  development. 

On  the  other  hand,  it  decreases  the  cost  of  frame 
generation  because  surface  masking  priority  is 
already  determined  and  thus  does  not  have  to  be 
repeated  in  other  CIG  routines  such  as  specular 
reflection  and  shadow  generation.’ 

The  list-directed  priority  scheme  uses  two 
distinct  methods  to  establish  the  priority  list. 
First,  all  surfaces  comprising  an  object  are 
listed  according  to  their  inherent  masking  prior¬ 
ity  within  the  object  such  that  no  surface  may 
mask  any  other  surface  listed  before  it,  regard¬ 
less  of  the  viewing  position.  Inherent  masking 
priority  restricts  surfaces  from  intersecting  each 
other  anywhere  but  at  their  edges.  For  a  convex 
object,  any  surface  order  is  acceptable  because  no 
surface  of  a  convex  object  will  ever  be  masked  by 
another  surface.  (Note:  Since  surfaces  are  one¬ 
sided,  surfaces  facing  away  from  the  viewing 
position  are  not  considered  masked.)  For  a  con¬ 
cave  object,  inherent  masking  priority  may  not 
exist,  in  which  case  the  object  can  be  divided 
into  smaller  objects  to  solve  the  masking 
problem. 

The  second  method  used  to  establish  the  pri¬ 
ority  list  of  all  surfaces  is  to  subdivide  the 
scene  into  objects.  Once  the  surfaces  within  each 
object  have  their  inherent  ordering  established, 
the  remaining  task  is  to  sort  the  objects.  To  do 
so,  a  tree  structure  was  implemented  which  can 
modularly  create  a  complex  scene  by  combining 
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simple  objects  to  form  more  complex  objects.  Or, 
as  Illustrated  In  Figure  2,  a  complex  scene  (such 
as  ships  at  sea)  can  be  broken  down  into  simple 
objects  and  geometric  primitives. 


SHIP  2  SHIP  1  OCCAM  2 


STARSOAftO  PORT  STERN  MIDSHIP  ROW  4 


Figure  2.  General  Tree  Representation  of 
Ship  Geometric  Data  Base 

To  fully  understand  the  tree  structure  used, 
definitions  of  some  terms  and  relationships  are 
necessary.  A  node  is  any  point  on  the  tree  and 
consists  of  one  or  more  objects  grouped  together. 

A  node's  descendants  (subnodes)  are  all  nodes  sub¬ 
ordinate  to  it.  The  node’s  closest  descendants 
(i.e.,  those  in  the  next  immediate  level)  are  des¬ 
ignated  as  the  node's  children.  A  node  is  consid¬ 
ered  a  parent  to  its  children  and  an  ancestor  to 
all  its  subnodes/descendants.  Nodes  sharing  a 
parent  are  siblings.  Thus,  in  Figure  2,  the  node 
TOTAL  SCENE  is  an  ancestor  to  all  nodes  of  the 
data  base  and  the  parent  of  its  children  SHIP  1, 
SHIP  2  and  OCEAN,  which  are  themselves  siblings. 

No  other  direct  relationships  are  defined  and  no 
significance  is  attached  to  the  relative  levels 
of  two  nodes  except  as  to  their  being  descendants, 
ancestors,  or  siblings.  For  example,  it  is  of  no 
consequence  that  the  nodes  for  the  PORT  side  of 
the  hull  and  the  MIDSHIP  section  above  the  deck 
are  both  at  level  4.  The  number  of  levels,  nodes 
of  the  tree,  and  children  of  any  node  are  unlim¬ 
ited.  A  node  with  no  subnodes  has  only  the  geo¬ 
metric  primitive  surfaces  as  its  children. 

For  each  node,  priority  tests  determine  the 
ordering  of  its  children.  Once  the  children 
(siblings)  have  been  ordered,  the  ordering  of  all 
their  descendants  (with  respect  to  the  siblings) 

Is  established  by  their  transitive  properties. 

For  example.  If  SHIP  1  masks  SHIP  2  for  a  partic¬ 
ular  eye  position,  so  do  all  descendants  of  SHIP  1. 
For  this  ordering  scheme  to  be  correct,  all  sib¬ 
lings  must  be  pairwise  linearly  separable  by  a 
separating  plane.  Stated  another  way,  a  node  may 
not  intersect  the  convex  cover  of  any  node  other 


than  its  ancestor  or  descendant.  If  such  an  inter¬ 
section  does  occur,  the  masking  relationship 
between  a  node's  siblings  and  its  descendants  is 
not  transitive  and  thus  the  masking  relationship 
between  unrelated  nodes  is  not  known.  For  some 
situations,  this  nonintersection  rule  may  be  re¬ 
laxed  during  data  base  development  if  it  is  known 
that  the  eye  will  be  restricted  to  certain  regions 
of  space,  such  as  the  upper  hemisphere  (i.e.,  above 
the  ground  or  ocean). 

Figure  3a  illustrates  a  simple,  2-0  case  in 
which  the  nodes  have  been  Incorrectly  formed  and 
thus  are  not  linearly  separable.  B(l)  and  B(2), 
separated  by  a  dashed  line,  have  been  combined  to 
form  their  parent  node  B.  However,  since  this 
parent  is  not  separable  from  node  A  by  a  plane, 
from  certain  viewing  positions  part  of  B  will  mask 
A  and  A  will  mask  some  other  part  of  B.  In  Figure 
3b,  A(2)  (which  is  B(2)  in  Figure  3a)  has  been 
combined  with  A(l)  to  form  parent  node  A.  In  this 
case,  nodes  A  and  B  are  linearly  separable  and  thus 
the  masking  relationships  are  transitive.  Hence, 
planes  to  separate  the  nodes  may  be  defined  within 
the  data  base  and  used  to  order  objects  once  view¬ 
ing  position  has  been  specified.  During  frame 
generation ,  A  and  B  will  be  ordered,  followed  by 
A( 1)  and  A(2).  If  further  subdivision  takes  place, 
ordering  continues  until  all  objects  are  ordered. 

At  that  point,  the  masking  problem  will  be  solved 
because  each  object’s  surfaces  were  previously 
listed  by  their  inherent  masking  priority. 


B  -  {B(1),B(2)} 


aui 


I  >12>  i  I 

a.  Nonlinearly  Separable  Nodes 


Figure  3.  Node  Formation 

The  geometric  data  base  is  stored  as  a  sepa¬ 
rate  directory  and  a  surface  list.  The  directory 
represents  each  node  by  a  unique  number  and 
contains  general,  relevant  information  associated 
with  each  node  (such  as  its  children's  node  numbers 
and  separating  planes  as  well  as  the  node's  local 
origin  and  bounding  box).  The  surface  list  con¬ 
tains  each  object's  surfaces  in  sequence,  along 
with  vertex  lists,  surface  and  vertex  normals. 
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material  descriptors,  and  an  associated  object 
node  number.  Whereas  the  surfaces  of  an  object 
must  be  listed  sequentially,  the  objects  them¬ 
selves  can  be  listed  In  any  order.  When  the  data 
base  is  converted  from  ASCII  to  binary  format, 
pointers  from  the  directory  to  the  first  surface 
of  each  object  In  the  surface  list  are  established 
to  provide  random  access  to  any  object. 


room  temperature,  a  substantial  amount  of  thermally 
emitted  radiation  is  in  the  8  to  12  urn  spectral 
band.  In  addition,  a  surface  reflects  thermal 
radiation  from  other,  nearby  emitters  such  as  the 
sky  and  water.  Thus,  the  total  emitted  energy 
from  a  surface  is  approximately  the  sum  of  its  own 
thermal  emission  and  the  reflected  energy  of  its 
surroundings. 


The  tree-structured  directory  offers  more 
than  just  a  solution  to  the  hidden-surface  problem. 
For  instance,  any  portion(s)  of  the  data  base  can 
be  viewed  by  specifying  desired  nodes.  By  default, 
the  total  scene  is  displayed.  In  addition,  the 
data  base  can  be  significantly  modified  through 
the  directory.  For  example,  by  changing  the  local 
origins,  objects  or  large  nodes  (such  as  an  entire 
ship)  can  be  moved.  However,  when  modifying  local 
origins,  the  separating  planes  between  sibling 
nodes  may  have  to  be  redefined,  and  motion  must  be 
restricted  so  that  nodes  do  not  intersect. 

Visual  and  Infrared  Toning 

Toning  is  the  CIG  phase  in  which  electromag¬ 
netic  models  are  implemented  to  predict  a  gray 
level  for  a  surface,  or  a  portion  of  it,  based  on 
the  sensor  model,  environmental  conditions,  and 
surface  characteristics.  The  toning  models  are 
essentially  independent  of  each  other  and  of  frame 
generation.  For  the  visible  band,  gray  levels 
correspond  to  the  percentage  of  total  light  re¬ 
flected.  For  the  infrared  ban;*,  e  y  levels 
correspond  to  the  emitted  and  reflected  infrared 
flux  density.  The  predicted  gray  level  replaces 
the  material  code  for  each  surface  or  each  vertex 
of  a  surface  in  the  geome.  ;ic  data  base. 

The  current  implemer. tation  tor  the  visual 
model  is  based  on  Lambert.  »  diffuse  reflection. 

For  this  model,  the  ton**  ited  is  independent 
of  rye  position  and  based  on  the  cosine  of  the 
angle  between  the  surface  normal  and  a  light 
source  (e.g.,  the  sun).  An  additional  component 
is  included  to  account  *  or  the  sky  as  a  light 
source. 

The  infrared  model  represents  a  more  compli¬ 
cated  situation  in  which  material  type,  internal 
temperature  loads,  air  temperature  and  humidity, 
and  the  contributions  of  surrounding  bodies  are 
major  factors  in  predicting  gray  levels.  A 
further  complication  is  that,  while  the  model  is 
diffuse  and  thus  independent  of  viewing  aspect, 
attenuation  due  to  range  can  be  significant.  The 
following  discussion  explains  the  infrared  model 
in  more  detail. 

Thermal  Emission.  The  radiant  emittance 
(the  power  per  unit  area  of  a  black-body  emitter) 
with  a  spectral  band  bounded  by  minimum  and 
maximum  wavelengths  Xj  and  X2,  respectively,  is 
given  by 


where  T  is  the  absolute  temperature  of  the  body, 
and  ci«C2  are  constants.  For  surfaces  at  or  near 


Infrared  Gray  Level  Prediction.  The  gray 
level  of  a  surface  is  directly  proportional  to  the 
surface's  radiant  emittance.  To  predict  a  sur¬ 
face's  radiant  emittance,  its  absolute  temperature 
at  any  arbitrary  time  must  first  be  predicted. 
Surface  temperature  is  a  function  of  the  thermal 
properties  of  the  material,  the  instantaneous 
temperature  of  the  surroundings,  and  the  previous 
temperatures  of  the  surroundings. 

In  the  infrared  toning  model,  an  environmental 
file  is  set  up  for  the  scene.  This  file  contains 
parameters  such  as  longitude,  latitude,  day  of  the 
year,  data  base  orientation  with  respect  to  North, 
and  hourly  air  temperatures  for  the  24  hours  prior 
to  the  "day"  the  ship  imagery  is  being  simulated. 
From  these  parameters,  the  radiative  and  convec¬ 
tive  heat  load  on  the  exterior  surface  at  any  time 
can  be  determined.  The  interior  surface  is 
assumed  to  be  exposed  to  a  constant  interior 
ambient  temperature.  The  conductive  path  between 
the  exterior  and  interior  surfaces  can  be  modeled 
as  an  equivalent  RC  electrical  network.  The 
surfaces  in  the  ship  data  base  are  assumed  to  be 
metal  plates,  and  the  equivalent  RC  network  model 
contains  eight  nodes,  as  shown  in  Figure  4. 
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Figure  4.  RC  Network  Model  for  Thermal 
Conduction  through  a  Metal 
Plate 


To  determine  the  precise  temperature  at  each 
node  requires  an  eighth-order  differential  equa¬ 
tion.  An  approximation  is  obtained  by  solving  a 
system  of  eight  first-order  linear  difference 
equations.  Thus,  a  reasonably  accurate  calcula¬ 
tion  can  be  obtained  for  the  exterior  temperature 
of  all  surfaces  in  the  data  base. 


Gray  levels  are  predicted  in  two  steps. 

First,  the  environmental  data  are  used  to  calcu¬ 
late  the  exterior  temperature  of  metal  plates 
whose  normals  are  oriented  in  twelve  standard 
directions,  and  a  look-up  table  is  generated. 
Second,  each  surface  in  the  data  base  is  addressed 
and  its  normal  vector  is  compared  to  the  twelve 
standard  directions  to  interpolate  the  surface 
temperature  and  add  in  the  reflected  infrared 
energy. 


-L. 
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Atmospheric  Attenuation.  Before  the  gray 
level  is  assigned,  the  effect  that  atmospheric 
attenuation  of  surface  emittance  has  on  the  gray 
level  is  calculated.  Atmospheric  attenuation 
depends  on  the  field-measurable  parameters  of  tem¬ 
perature,  relative  humidity,  range,  and  visibility. 
A  scaled-down  version  of  the  10WTRAN  4  atmospheric 
transmission  model  developed  at  the  Air  Force 
Geophysics  laboratories  is  used.  The  maritime 
aerosol  model  and  a  horizontal  transmission  path 
are  assumed.  The  gray  level  assigned  is  based  on 
the  attenuated  emittance  "seen"  by  the  sensor. 

Frame  Generation 

User- Selected  Parameters.  A  number  of  param- 
eters/options  are  available  to  the  user  for 
generating  imagery  in  many  formats  and  for  many 
conditions.  The  user  can  select: 

1.  Oata  base  and  associated  level  of  detail 

2.  Eye  position  in  x,y,z  scene  coordinates 

3.  Aim  point  (i.e. ,  image  center)  in  x,y,z 
scene  coordinates 

4.  Frame  size--number  of  rasters,  number 
of  pixels  per  raster 

5.  Field  of  view-horizontal  and  vertical 
in  degrees 

6.  Rotation  in  degrees 

7.  Special  features  for  earth  curvature, 
smooth-surface  shading,  and  edge 
smoothing. 

The  frame  size  and  field  of  view  chosen  dictate 
the  sampling  resolution  per  pixel.  Field  of  view 
is  also  inversely  proportional  to  magnification. 
For  a  given  frame  size,  objects  remaining  in  the 
field  of  view  will  subtend  more  pixels  as  the 
selected  field  of  view  decreases.  For  the  peri¬ 
scope  modeled,  magnification  of  IX  corresponds  to 
a  48°  field  of  view,  6X  to  8°. 

Geometric  Processing.  Geometric  processing 
is  a  preprocessing  step  to  raster  processing.  It 
comprises  all  operations  by  which  the  potentially 
viewable  surfaces  are  ordered  and  oriented  rela¬ 
tive  to  the  eye. 

As  explained  earlier  (see  Geometric  Data  Base 
Modeling),  objects  are  ordered  through  tree  tra¬ 
versal,  during  which  sibling  nodes  are  compared 
via  separating  planes.  For  a  pairwise  comparison, 
the  node  on  the  same  side  of  the  separating  plane 
as  the  eye  has  priority  over  the  other  node. 

Since  each  object's  surfaces  are  presorted,  all 
surfaces  are  ordered  when  all  objects  have  been 
ordered. 

All  surfaces  are  then  clipped  by  the  planes 
of  the  field  of  view  using  a  reentrant  polygon 
clipper."  The  clipping  process  can  be  rapidly 
accelerated  by  determining  whether  the  bounding 
boxes  of  the  objects  are  totally  inside  or  outside 
the  field  of  view.  If  the  effects  of  earth  curva¬ 
ture  are  to  be  simulated  (so  that  the  ship  drops 
off  the  horizon),  the  surfaces  are  also  clipped 
against  an  ocean  plane. 


Finally,  all  remaining  surfaces  are  translated 
and  rotated  relative  to  the  eye  coordinate  system, 
in  which  the  origin  is  at  the  eye  and  one  of  the 
major  axes  lies  along  the  line  of  sight.  The  re¬ 
sulting  edges  are  stored  in  a  general  edge  list  for 
raster  processing. 

Note  that  the  surface  vertices  are  not  trans¬ 
formed  to  2-D  screen  coordinates  (normally  a 
perspective  transformation)  at  this  stage.  Instead, 
the  transformation  is  performed  during  raster  pro¬ 
cessing  because  a  nonlinear  projection  such  as  a 
cylindrical  or  spherical  projection5  requires  the 
original  3-D  information.  For  the  CIG  system 
discussed  in  this  paper,  only  the  perspective  pro¬ 
jection  is  used. 

Raster  Processing.  The  list-directed  priority 
scheme  considerably  simplifies  raster  processing  in 
that  no  further  testing  is  needed  to  determine 
which  surfaces  may  mask  other  surfaces.  Frame 
generation  proceeds  one  raster  at  a  time,  top  to 
bottom.  As  each  raster  is  processed,  the  points 
at  which  all  edges  enter  and  exit  and  raster  are 
maintained.  The  edges  are  then  projected  onto 
the  screen. 

Each  raster  is  generated  pixel  by  pixel,  left 
to  right.  For  each  pixel  processed,  a  stack  is 
maintained  and  updated  to  indicate  which  surfaces 
in  the  pixel,  delimited  by  their  edges  in  the 
raster,  are  potentially  visible.  If  no  edges  of 
the  top  surface  in  the  stack  lie  in  the  pixel,  the 
pixel  is  completely  shaded  by  that  surface.  If  the 
surface  has  a  uniform  shade,  the  predicted  surface 
tone  is  used.  If  the  surface  shade  varies  between 
vertices,  Gouraud  shading6  is  applied. 

Whenever  edges  are  visible  in  the  pixel, 
aliasing  effects  are  reduced  by  an  edge-smoothing 
technique  which  assigns  a  shade  to  the  pixel  based 
on  the  relative  amount  of  area  of  each  surface  in 
the  pixel.  The  area  of  each  surface  is  calculated 
by  a  simple  sunmation  of  triangles  and  rectangles 
outlined  by  the  surface  edges  and  borders  of  the 
pixel.  This  technique  does  not  reduce  aliasing 
effects  as  completely  as  Catmull's  integrator 
algorithm7  because  it  only  estimates  portions  of 
any  surface  other  than  the  top  surface  that  are 
actually  visible  in  the  pixel.  However,  this 
approach  requires  less  computation  and  signifi¬ 
cantly  reduces  the  stairstep  effect  observed  in 
raster  graphics. 

SHIP  MODELS  AND  IMAGERY 

The  major  cues  for  recognizing  ships  are  the 
edge  content  and  distinguishable  shapes  produced  by 
contrasts  in  gray  shades.  Although  texture  is  an 
important  recognition  cue  in  many  simulations,  it 
is  of  little  importance  for  ships  because  they 
usually  have  dull  gray  or  black  overcoats  of  paint. 
The  ship's  silhouette  provides  the  primary  sources 
of  contrast;  however,  certain  areas  of  the  ship 
provide  other  contrast  sources  (e.g.,  hull  con¬ 
cavities  and  "hot  spots"  in  Infrared  simulations). 

The  overriding  goal  of  data  base  modeling  was 
to  produce  the  ship's  general  outline  along  with 
those  portions  of  the  superstructure  which  add 
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detail  to  the  outline.  In  addition,  we  modeled 
major  objects  which  blended  in  with  the  rest  of  the 
ship  from  a  broadside  view  but  were  prominent  from 
a  quartering  aspect.  In  line  with  the  hierarchy 
illustrated  in  Figure  2,  data  bases  were  developed 
by  dividing  each  model  into  its  major  components 
and  then  subdividing  until  all  objects  of  interest 
were  defined  and  their  surfaces  could  be  listed  by 
inherent  masking  priority. 

Emphasis  was  also  placed  on  modeling  objects 
so  as  not  to  display  unwanted  edges,  such  as  those 
generated  by  planar  patches  used  to  simulate 
curved  surfaces.  Smooth  shading  was  attempted  on 
selected  portions  of  the  ship  to  demonstrate  its 
effects. 

Data  bases  for  each  ship  range  in  level  of 
detail  from  approximately  500  to  4000  potentially 
visible  edges.  Data  bases  of  high  level  of  detail 
were  developed  first,  followed  by  data  bases  of 
progressively  lower  levels.  This  approach  was 
based  on  the  assumption  that  it  is  easier  to  remove 
than  to  add  detail.  The  three  methods  implemented 
to  achieve  the  lower  levels  of  detail  were 
1)  planar  patches  were  reduced  in  number  and  in¬ 
creased  in  size  so  that  less  or  no  curved  surfaces 
were  simulated,  2)  objects  or  surface  detail  not 
contributing  to  the  broadside  silhouette  or  other 
major  areas  of  contrast  were  eliminated,  and 
3)  small  objects  were  either  removed  or  grouped  to 
form  major,  simplified  objects. 

Figures  5  through  8  and  Figures  9  and  10  are, 
respectively,  visual  images  of  the  Kashin  and  Kiev 
high- level -of- detail  models.  These  images  illus¬ 
trate  the  complete  3-D  nature  of  the  models.  The 
Kashin  model  consists  of  63  objects  and  1150  sur¬ 
faces;  the  Kiev  model  of  103  objects  and  2300 
surfaces.  The  images  were  generated  on  the  VAX-11/ 
780  computer  at  the  Experimental  Computer  Simula¬ 
tion  Laboratory  of  NTEC,  and  were  photographed  with 
a  Di corned  D-47  image  recorder.  Options  for  edge 
smoothing  and  smooth  shading  were  used. 

For  the  Kashin,  the  midsection  was  emphasized 
because  of  its  prominent  structures  such  as  the 
smokestack  and  radar  towers.  In  Figure  8,  the  main 
radar  tower's  complex  tubular  structure  illustrates 
the  detail  that  can  be  portrayed.  In  addition,  the 
hull  was  emphasized  because  of  its  distinctive 
shape  and  the  line  that  runs  along  much  of  it,  both 
of  which  are  significant  recognition  cues.  (The 
hulls  of  U.S.  destroyers  are  flat  from  bridge  to 
stem. ) 

For  the  Kiev,*  importance  was  placed  on  model¬ 
ing  the  dominant  superstructure  and  the  irregular, 
asymmetric  hull  with  its  concave  sections.  As  was 
done  for  the  Kashin,  radar  antennas  were  modeled 
because  they  are  distinguishable  features.  The 
top-sail  radar  (dark  object  above  the  bridge)  was 
modeled  with  especially  high  detail  using  a  grid- 
work  of  tubes. 


Although  the  figures  show  the  Kiev  as  equal 
in  size  to  the  Kashin,  the  Kiev  model  is  actually 
twice  as  large.  The  field  of  view  for  the  Kiev 
images  was  halved. 


RECOMMENDATIONS 

Whereas  the  CIG  software  at  TSC  needed  only 
minor  modifications  to  handle  the  large  geometric 
data  bases  and  the  maritime  environment,  the  geo¬ 
metric  data  base  modeling  required  much  effort 
because  of  the  manual,  labor-intensive  techniques 
used.  Ship  measurements  were  made  from  line  draw¬ 
ings  and  entered  into  the  geometric  data  base  via 
keyboard.  Most  of  the  validation  consisted  of 
visually  inspecting  sample  computer-generated 
imagery.  Tasks  such  as  ordering  surfaces  by  their 
inherent  masking  priority  and  choosing  the  detail 
to  be  used  for  modeling  objects  were  generally 
performed  by  the  modeler. 

An  interactive  data  base  editor  with  the 
proper  graphics  hardware  could  significantly  re¬ 
duce  the  throughput  time  for  developing  and  vali¬ 
dating  a  geometric  data  base.  Data  could  be 
entered  via  digitizer  tablet  or  keyboard  and  dis¬ 
played  on  a  vector  graphics  terminal.  With  cursor 
control  and  a  convenient  set  of  command  capabili¬ 
ties,  the  modeler  could  manipulate  the  data  on  a 
vertex,  surface,  or  node  level  to  achieve  any 
desired  geometric  configuration.  Operations  useful 
to  the  modeler  could  range  in  complexity  from 
simple  translation  and  rotation  routines  to  auto¬ 
matic  surface  ordering  within  an  object,  and  even 
to  automatic  object  subdivision  to  resolve  surface 
masking-priority  conflicts. 
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Figure  7.  Quartering  View  of  Kashin 


Figure  8.  Close-Up  of  Kashin  Midsection 
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Figure  10.  Aerial  View  of  Kiev 
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A8STRACT 


The  problem  of  operational  user  acceptance  of  naval  training  R&D  studies  and  training  devices 
is  presented.  A  number  of  constraints  on  acceptance  are  described,  including  deficiencies  in 
motivational  conditions,  deficiencies  in  social  role  assignments,  deficiencies  in  official 
organizational  policy  and  structure,  inadequate  defense  R&D  contracting  methods,  lack  of 
integration  of  the  user  into  the  trainer  system  acquisition  process,  other-than-rational  user 
responses  to  R&D  studies  in  training,  and  deficiencies  in  training  device  design.  A  prelimin¬ 
ary  model  of  the  acceptance  process  is  presented.  Finally,  the  degree  of  physical  fidelity 
necessary  for  operational  acceptance  and  training  effectiveness  is  discussed.  It  is  concluded 
that  this  paper  and  related  recent  work  form  a  basis  for  the  development  of  alternative  ap¬ 
proaches  to  solving  the  user  acceptance  problem. 


INTRODUCTION 

Scientific  research  and  development  (R&D)  is  a 
major  institutionalized  means  for  creating  im¬ 
provements  in  military  technology.  The  R&D  com¬ 
munity  creates  these  improvements  not  only  by 
constantly  advancing  the  technological  data  base, 
but  also  by  introducing  that  technology  into 
fleet  operations.  In  fact,  the  force  for  change 
from  "technology  push"  (12)  by  R&D  agencies  has 
been  traditionally  stronger  in  generating  opera¬ 
tional  requirements  than  that  from  "requirements 
pull"  by  the  operational  fleet  who  formally  es¬ 
tablish  Navy  requirements  (28).  Training  techno¬ 
logy  has  also  advanced  rapidly  in  the  last  decade. 
However,  there  is  considerable  evidence  that 
training  organizations  are  reluctant  to  accept 
the  introduction  of  R&D  studies.  In  addition, 
acceptance  of  specific  training  devices  is  a 
major  problem.  There  has  been  very  lit  ,  in¬ 
vestigation  of  the  factors  influencing  tne  ac¬ 
ceptance  of  new  training  methods  introduced  by 
R&D  studies.  R&D  has  generally  fulfilled  its 
role  in  creating  the  potential  for  useful  changes 
in  operational  training,  but  has  not  developed 
an  adequate  model  to  assist  in  how  change  should 
be  managed.  Basic  organizational  and  human  re¬ 
sistance  to  change  has  remained  stable,  whereas 
the  rate  of  change  itself  has  greatly  increased. 

A  major  task  of  training  R&D,  therefore,  is  to 
develop  managerial  methods  for  the  successful 
planning  and  implementing  of  changes  in  train¬ 
ing  technology. 

Importance  of  the  User  Acceptance  Problem. 

Improved  acceptance  is  especially  critical  now 
for  several  reasons: 

1.  Significant  advances  in  effective 
training  technology  remain  to  be  incorporated 
into  the  Fleet.  Computer  training  technology 
(3)  and  maintenance  training  (23)  advancements 
are  two  major  areas  of  "gaps"  between  available 
technology  and  its  applied  use.  In  addition, 
further  R&D  studies  need  to  be  performed  to 


evaluate  and  improve  currently  implemented  train¬ 
ing  systems. 

2.  A  variety  of  low-cost,  low-fidelity 
approaches  to  training  system  design  are  now 
under  development.  Yne  degree  of  physical 
fidelity  (real  ism)  has  been  shown  to  be  the 
single  most  important  variable  in  gaining  user 
acceptance  (7).  However,  the  current  practice  of 
"buying"  user  acceptance  of  simulation  training  at 
the  (high)  cost  of  increased  physical  fidelity  is 
being  questioned.  Thus,  the  loss  of  acceptance 
from  fidelity  sources  requires  improved  accep¬ 
tance  conditions  from  areas  other  than  fidelity. 
This  issue  will  be  discussed  in  a  subsequent 
section. 

3.  There  is  an  increasing  need  to  intro¬ 
duce  and  to  have  accepted  training  improvements 
as  early  as  possible.  Over  the  past  decade  the 
time  tu  acquire  a  major  weapon  system  (and  re¬ 
lated  training  materials)  has  steadily  increased 
(11).  In  the  current  environment  of  internation¬ 
al  tension,  it  is  imperative  that  the  user  be  in 
a  position  to  move  the  acquisition  process  along 
and  to  incorporate  training  improvements  as 
rapidly  as  feasible. 

4.  Training  technology  will  be  increas¬ 
ingly  integrated  into  ongoing  fleet  operational 
systems .  With  the  advent  of  portable,  rugged, 
and  compact  training  systems  has  come  the  possi¬ 
bility  of  training  aboard  ship  (called  onboard 
training  or  OBT)  --  either  in  the  form  of  origin¬ 
al  or  refresher  training  (e.g.,  landing  refresher 
trainers  in  the  pilot  ready  room  as  currently 
envisioned  by  NAVTRAEQUIPCEN  -  see  25).  Recent 
shipboard  weapon  system  design  with  new  class 
destroyers  includes  provisions  for  embedded 
training  devices  used  to  support  team  training. 

The  success  of  the  OBT  concept  will  depend  criti¬ 
cally  on  acceptance  of  this  concept  by  fleet 
personnel . 

5.  An  especially  critical I  protlem  of 
accepting  flight  simulators  as  flight  hour 
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substitution  devices  now  exists.  In  a  document 
(8)  issued  by  the  Chief  of  Naval  Operations  (CNO), 
authorization  and  plans  for  achieving  a  25U  re¬ 
duction  in  Navy  flight  hours  (and  huge  fuel 
savings)  by  the  end  of  FY  81  were  stated.  The 
flight  simulator  was  viewed  as  a  potential 
flight  substitution  device  by  higher  conmand 
echelons  of  the  Navy.  Local  training  commands, 
however,  have  continued  to  view  the  simulator  as 
a  means  for  augmenting  rather  than  replacing 
aircraft  training.  This  discrepancy  between 
perceptions  of  the  purpose  of  the  device  is  a 
major  reason  for  the  documented  underutilization 
of  aviation  training  devices  and  the  consequent 
failure  to  demonstrate  cost  savings  through 
flight  hour  reductions  (Navy  Audit  Review,  24). 

6.  Operational  tasks  and  equipment  are 
becoming  increasingly  complex  (14)  and  human 
error  correspondingly  probable  and  costly~  Thus, 
greater  reliance  of  training  substitutes  for 
operational  environments  will  be  necessary,  for  a 
variety  of  both  high-cost  and  low-cost  forms  of 
training. 

Focus  of  this  Report.  The  focus  here  is  on  ac- 
ceptability  of  new  training  methods  by  training 
system  users.  Acceptance  of  training,  however, 
is  regarded  as  a  tri-service  problem  and  a  con¬ 
ceptual  model  of  the  acceptance  process  needs  to 
be  developed  for  tri-service  use  (see  below). 

A  second  major  focus  is  on  the  factors  support¬ 
ing  the  introduction  of  R&D  studies  into  train¬ 
ing  organizations,  in  contrast  to  subsequent 
testing,  installation  and  institutionalization 
of  the  change  into  ongoing  fleet  training  cycles. 
Obviously,  successful  acceptance  at  the  beginning 
will  make  subsequent  implementation  much  more 
likely.  Although  this  report  will  often  illu¬ 
strate  the  process  of  acceptance  in  terms  of  an 
R&D  study  with  a  training  device,  the  successful 
introduction  of  R&D  studies  in  all  operational 
training  areas  is  of  prime  concern.  Where  de¬ 
vices  are  mentioned,  they  will  be  aviation  train¬ 
ing  devices  (ATDs)  due  to  the  authors'  experience 
in  this  area  and  the  high  cost  of  these  trainers. 
Underutilization  and  even  non-utilization  of  ATDs 
has  been  documented  for  both  Air  Force  (7)  and 
Navy  (20)  devices.  These  two  sources  are  the 
only  studies  that  have  systematically  reviewed 
the  effects  of  different  user  attitudes  and  de¬ 
vice  features  on  utilization  of  ATDs.  Many  of 
the  observations  made  by  these  earlier  reports 
regarding  ATDs  were  supported  herein  as  also  be¬ 
ing  relevant  to  the  acceptance  of  R&D  studies. 
Additional  observations  have  been  provided  in  the 
present  paper  regarding  acceptance  of  R&D,  many 
of  which  would  seem  to  apply  as  well  to  ATD  ac¬ 
ceptance.  Further,  a  model  for  user  acceptance 
of  R&D  has  been  developed  which  represents  a  be¬ 
ginning  effort  to  integrate  theoretical  litera¬ 
ture  from  the  general  area  of  organizational 
change  with  the  earlier  and  present  observations 
from  ATD  and  R&D  field  implementations. 

Perspectives  on  Acceptance  and  Change. 

1.  Who  has  the  acceptance  problem?  There 
is  a  natura)  tendency  to  focus  on  the  operation- 
al  user  when  thinking  about  acceptance  of  R&D. 

If  R&D  studies  or  results  are  not  accepted,  the 
user  may  be  labeled  as  resistant  to  change.  The 
view  here  is  that  the  user  can  take  neither 


credit  for  acceptance  nor  blame  for  the  rejection 
of  R&D  efforts.  Instead,  acceptance  depends  on 
a  large  number  of  factors  influencing  training 
at  the  individual,  group,  organizational ,  and  CNO 
levels.  Acceptance  is  a  training  system  output 
and  therefore  the  system  should  be  the  target  for 
change. 

A  major  purpose  of  this  report  is  to  identify  a 
number  of  general  constraints  on  acceptance, 
only  some  of  which  exist  at  the  operational  user 
level . 

2.  Is  high  user  acceptance  the  goal  of 
R&D  efforts?  User  acceptance  is  a  continuous 
variable  ranging  between  the  extremes  of  complete 
rejection  of  an  innovation  to  complete  acceptance. 
It  is  apparent  that  even  the  most  effective  and 
economical  training  program  or  device  can  have  no 
value  if  it  is  rejected  (unused).  However,  com¬ 
plete  acceptance  of  a  training  method  is  not 
necessarily  desirable  either,  since  acceptance 
also  may  be  based  on  reasons  other  than  effec¬ 
tiveness  and  economy.  Caro,  Shell nutt,  and 
Spears  (7),  e.g.,  have  made  an  important  distinc¬ 
tion  between  acceptance  of  a  device  as  used  and 
acceptance  of  a  device  as  it  was  designed  to  be 
used.  In  flight  training,  there  is  considerable 
evidence  that  even  when  devices  are  well  accepted, 
their  capabilities  are  often  underutilized  (29). 
Operational  Flight  Trainers  (OFTs),  e.g.,  are 
often  highly  accepted  only  as  Cockpit  Procedures 
Trainers  (CPT).  The  primary  goal  of  R&D  efforts 
is  not  on  maximizing  user  acceptance,  but  on 
maximizing  organizational  effectiveness  and  economy 
of  the  training  organization. 

3.  Is  the  proposed  change  always  desir¬ 
able?  The  perspective  taken  here  is  that,  in 
specific  R&D  studies  planned  with  a  particular 
user  group,  change  introduced  by  the  R&D  should 
only  be  considered  after  careful  diagnosis  of  the 
existing  training  program.  Thus,  R&D  scientists 
are  not  to  be  labeled  only  as  "change  agents"  but 
also  as  "interventionists",  a  tern  used  by 
Argyris  (2)  to  characterize  a  person  who  inter¬ 
venes  to  diagnose  an  organization's  effective¬ 
ness,  but  who  does  not  necessarily  follow  through 
as  a  change  agent. 

Some  R&D  studies  are  ill-conceived;  appro¬ 
priate  diagnosis  may  reveal  that  there  is  no 
training  problem  as  originally  conceived,  or  that 
the  proposed  R&D  effort  is  irrelevant  to  correct¬ 
ing  the  identified  problem,  or  that  the  original 
R&D  problem  is  not  significant.  Or  there  may 
exist  a  variety  of  constraints  on  acceptance 
which  make  the  study  infeasible.  These  con¬ 
straints  on  acceptance  are  the  most  compelling 
reasons  for  deciding  not  to  introduce  change, 
since  almost  all  training  programs  could  stand 
some  improvement  and  almost  all  R&D  studies  can 
be  (and  usually  are)  adapted  to  become  more  rele¬ 
vant  to  the  needs  of  the  local  operational  group. 
The  rational  approach  to  R&D  planning  Includes 
the  option  of  not  doing  R&D  studies  where  a  var¬ 
iety  of  constraints  on  acceptance  of  change  makes 
the  study  exceptionally  risky  or  infeasible. 

These  constraints  which  exist  at  all  levels  of 
the  Navy  training  conmand  structure,  will  be  de¬ 
scribed  in  detail  below. 

4.  To  increase  acceptance,  should  the  man 
or  the  machine  be  changed?  In  traditional  human 
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factors  engineering,  there  is  a  focus  on  how  roan 
interacts  with  machines  (e.g.,  training  devices). 

To  "perfect"  the  interaction,  a  scientist  can  at¬ 
tempt  to  change  either  the  man  or  the  machine. 

One  approach  emphasizes  changing  the  man  through 
"attitude  engineering"  to  increase  acceptance  of 
device  features  existing  in  current  trainers.  An 
apparently  more  common  approach  is  to  focus  on 
the  design  characteristics  of  the  training  machine, 
especially  in  terms  of  physical  fidelity  or 
realism.  Both  approaches  need  to  be  included  in 
any  comprehensive  framework  of  the  acceptance 
process.  In  addition,  the  framework  must  include 
"man-man"  interactions  among  users,  sponsers,  and 
R&D  personnel  throughout  the  entire  training  sys¬ 
tem  acquisition  process,  from  introduction  of 
initial  R&D  studies,  through  design  reviews,  ini¬ 
tial  device  evaluations,  and  subsequent  training 
support.  Grace  (15),  in  her  1979  presidential 
address  before  the  Hunan  Factors  Society,  has 
pointed  out  that  human  factors  specialists'  have 
traditionally  overemphasized  machines,  devices, 
and  systems,  relative  to  human  interaction  prob¬ 
lems  in  the  design,  development,  and  use  of  these 
machines. 

5.  Is  a  scientific  approach  to  the  prob¬ 
lem  of  user  acceptance  possible?  Some  may  resist 
the  idea  that  science  can  be  used  to  analyze  and 
actually  influence  something  so  apparently  nebu¬ 
lous  and  subjective  as  "acceptance".  Although 
the  state  of  science  is  not  well  developed  in 
the  acceptance  area,  there  are  some  theoretical 
and  empirical  bases  for  influencing  levels  of 
acceptance.  An  initial  step  in  the  scientific 
approach  would  be  to  document  the  extent  of  vari¬ 
ation  of  acceptance  found  in  various  aspects  of 
naval  training.  Factors  that  influence  accep¬ 
tance  would  subsequently  be  identified,  de¬ 
scribed,  and  prioritized  for  different  applica¬ 
tions.  Then  improved  metrics  for  these  factors 
would  be  generated.  A  conceptual  framework 
consisting  of  these  factors  and  their  relation¬ 
ships  to  one  another  would  be  developed  to  under¬ 
stand  the  process  of  acceptance  and  to  use  as  a 
basis  for  predicting  acceptance  levels  in  parti¬ 
cular  situations.  Some  of  these  factors  have 
predictable,  but  uncontrollable  consequences  on 
acceptance  levels;  other  factors  are  controllable 
by  those  with  influence  in  the  training  command 
structure.  The  R&D  conmunity  can  become  more 
proficient  at  managing  the  introduction  of  train¬ 
ing  innovations  by  applying  those  factors  that 
can  be  controlled  to  influence  acceptance. 

Organization  of  this  Report.  The  remainder  of 
this  paper  is  organized  into  four  major  sections: 

1.  Terminology  of  user  acceptance 

2.  Major  constraints  on  acceptance 

3.  Conceptual  framework  for  acceptance 

4.  Issues  in  user  acceptance 

TERMINOLOGY  OF  USER  ACCEPTANCE 

Definition  of  the  User.  The  usual  focus  on  ac- 
ceptance  is  on  the  operational  user  —  the  level 
at  which  a  training  device  or  system  is  operated 
or  maintained.  However,  in  terms  of  the  success¬ 
ful  introduction  of  an  R&D  study,  acceptance 
must  exist  on  at  least  two  additional  levels  — 


the  sponsor  or  funding  level,  and  the  technical 
or  scientific  level  responsible  for  i  (mediate 
management  of  the  R&D  study.  If  acceptance  depends 
upon  several  levels  of  users,  an  overall  metric 
of  acceptance  would  include  input  from  each  of 
these  primary  users.  The  role  of  each  user  would 
reflect  his  major  expertise  and  needs  at  each 
level.  Thus,  acceptance  would  be  maximal  when 
the  R&D  work  was  seen  as:  (1)  credible  by  the 
operational  user  who  uses  his  operational  exper¬ 
tise  to  judge  the  training  value  of  the  R&D  work; 
(2)  affordable  by  the  sponsor  user  who  uses  his 
cost-benefit  analytic  expertise  to  judge  the 
value  of  the  proposed  work  in  terms  of  funding 
requirements  of  competing  programs;  and  (3) 
researchable  by  the  scientist  user  who  uses  his 
technical  expertise  to  judge  the  scientific  value 
of  the  work.  Unless  the  subsequent  text  refers 
specifically  to  the  sponsor  or  the  scientific  user, 
it  should  be  assumed  that  the  user  is  the  opera¬ 
tional  one. 

Definition  of  Acceptance.  At  the  scientific  user 
level,  there  are  well  developed  rules  or  validi¬ 
ty  factors  for  accepting  as  worthwhile  a  proposed 
R&D  study  (cf.,  6).  At  the  sponsor  user  level, 
there  also  exist  established  procedures,  federal 
regulations  as  well  as  statutory  proscriptions 
to  judge  the  acceptability  of  a  study  in  terms 
of  funding  limits,  appropriate  funding  appropria¬ 
tions,  service  priorities  within  funding  cate¬ 
gories,  fiscal  year  time  constraints  on  obliga¬ 
tion,  expenditure  of  funds,  etc.  At  the  oper¬ 
ational  user  level,  however,  the  criteria  for 
acceptability  typically  are  not  based  on  scienti¬ 
fic,  regulatory,  or  statutory  grounds.  The 
criteria  for  acceptance  are  based  mainly  on  a 
tradition  which  requires  the  user  to  make  primar¬ 
ily  subjective  determinations  of  training  value. 
Subjective  opinion  will  undoubtedly  continue  as 
a  primary  measure  of  acceptance.  Although  high 
subjective  acceptance  of  training  methods  does 
not  guarantee  optimal  training  value,  it  is  cer¬ 
tainly  true  that  an  absence  of  subjective  support 
will  reduce  the  use  of  methods  that  may  be  high¬ 
ly  acceptable  to  both  scientific  and  sponsor 
level  users. 

A  complete  operational  definition  of  acceptance 
will  include  all  three  user  levels.  In  addition, 
within  each  user  level,  there  may  be  several 
levels  of  acceptance  of  change  (as  modified  from 
Hersey  &  Blanchard,  16)  that  can  take  place: 


1. 

Individual  knowledge  of  an  alternative 

2. 

training  method. 

Individual  attitudinal  cornnitment  to 

3. 

the  alternative. 

Individual  behavioral  comni tment  to 

4. 

the  alternative. 

Group  behavioral  coemi tment  to  the 

5. 

procedures  called  for  by  the  alterna¬ 
tive. 

Organizational /Institutional  policy 

commitment  to  the  alternative. 

This  expanded  definition  of  acceptance  adds  two 
Important  perspectives  to  the  attempts  of  R&D  to 
Introduce  behavioral  changes  In  training  prac¬ 
tices:  (1)  failure  to  obtain  behavioral  change, 
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at  either  the  Individual  or  group  level  does  not 
necessarily  mean  that  no  change  in  knowledge  or 
attitude  has  taken  place.  These  "lower  level" 
changes  in  knowledge  and  attitude  can  provide  an 
important  basis  for  subsequent  attempts  to  ob¬ 
tain  behavioral  conmitments;  (2)  even  change  in 
the  form  of  behavioral  comnitment  may  fail  to 
have  longer  term  Impact  if  the  innovation  does 
not  become  institutionalized  into  the  organiza¬ 
tion's  training  policies. 

Finally,  each  of  these  levels  of  acceptance  are 
continuous  variables.  Thus,  the  RAO  scientist 
should  not  expect  to  obtain,  nor  the  user  feel 
compelled  to  provide,  unqualified  acceptance. 

It  is  essential  that  the  user  understand  that 
his  "acceptance"  of  an  RAD  study  does  not  neces¬ 
sarily  constitute  complete  agreement  with  the 
proposed  change.  He  should  accept,  however,  the 
responsibility  to  honestly  attempt  to  evaluate 
the  training  consequences  of  the  implemented 
change. 

CONSTRAINTS  ON  OPERATIONAL  USER  ACCEPTANCE 

An  analysis  of  the  acceptance  problem  from  a  sys¬ 
tems  point  of  view  reveals  several  kinds  of  con¬ 
straints  that  mitigate  or  even  prevent  acceptance 
of  an  RAD  study  or  of  an  ATD.  These  constraints 
exist  at  all  levels  of  human  behavior,  including 
the  personal,  social,  and  organizational  levels 
of  the  Navy.  Listed  below  are  several  major 
kinds  of  constraints  on  acceptance.  Although 
there  is  some  overlap  between  these  various  con¬ 
straints,  they  do  appear  to  circumscribe  the 
classes  of  limitations  that  would  need  to  be  con¬ 
sidered  (and  further  developed)  in  any  comprehen¬ 
sive  treatment  of  the  acceptance  process. 

1.  Deficiencies  in  User  Motivational 
Conditions. 

2.  Deficiencies  in  User  Role  Assignments. 

3.  Deficiencies  in  Official  Navy  Policy 
and  Structure. 

4.  Inadequate  Defense  RAD  Contracting 
Methods. 

5.  Inadequate  Integration  of  the  User  into 
the  Weapon  Systems  Acquisition  Process 
through  Participative  Management  (PM). 

6.  Other-than-Rational  User  Responses  to 
RAD  Studies  and  to  ATDs. 

7.  Deficiencies  in  Training  Device  Design. 

Deficiencies  in  User  Motivational  Conditions. 

It  is  axiomatic  that  there  must  be  some  personal 
motivational  basis  for  the  acceptance  of  an  RAD 
study  or  a  new  ATD.  Adams  (1)  has  recently 
stated  that  one  of  the  five  major  learning  prin¬ 
ciples  on  which  ATDs  are  (or  at  least  should  be) 
designed  and  used  is  trainee  motivation  to  per¬ 
form  the  task.  He  refers  to  this  principle  as 
"customer  acceptance".  While  it  is  true  that  the 
trainee's  motivation  to  practice  Is  important, 
the  more  crucial  training  element  for  both  the 
acceptance  of  RAD  studies  and  the  use  of  an  ATD 
Itself  appears  to  be  the  Instructor.  If  the  in¬ 
structor  fails  to  accept  the  Innovation,  it  will 
not  reach  the  trainee.  Moreover,  the  instruc¬ 


tor's  attitudes  and  role  modeling  play  crucial 
roles  in  motivating  the  trainee  to  use  an  ATD 
(cf.,  7).  It  is  also  the  Instructor  who  is  in  a 
position  to  coimit  himself  to  implementing  the 
training  methods  proposed  by  the  RAD. 

An  analysis  of  the  motivational  conditions  common¬ 
ly  found  when  Instructors  participate  in  RAD  pro¬ 
jects  reveals  serious  deficiencies.  These  in¬ 
clude: 

1.  Little  or  no  official  recognition  (via, 
e.g.,  fitness  reports,  achelvement  awards)  is 
given  for  Instructor  contributions  to  RAD.  As  a 
result,  the  RAD  is  perceived  as  a  low  organiza¬ 
tional  priority  and  unrelated  to  their  position 
in  the  Navy. 

2.  Incentives  other  than  official  recogni¬ 
tion  are  also  deficient.  Although  the  instructors 
are  essential  to  implement  the  study,  they  re¬ 
ceive  little  credit  for  successful  outcomes. 

3.  Participation  in  RAD  often  increases 
the  total  workload  of  users  such  that:  (a)  ab¬ 
normally  long  and/or  time-stressed  periods  are 
required  to  accomplish  the  job;  (b)  participation 
in  RAD  threatens  performance  on  other  tasks  (of¬ 
ten  more  critical  in  terms  of  fitness  reports, 
advancements,  etc.)  due  to  time  taken  from  these 
tasks  for  RAD  assignments.  Thus,  RAD  may  be 
viewed  by  users  as  a  work  overload  as  well  as  a 
threat  to  their  position  in  the  Navy. 

4.  RAD  studies  are  sometimes  perceived  by 
instructors  as  an  assault  to  their  professional 
status.  First,  there  are  ways  to  significantly 
improve  training  which  users  feel  they  should 
have  Implemented  but  did  not.  Thus,  RAD  can 
threaten  the  users'  status  as  training  designers. 
Secondly,  some  training  operations  (e.g.,  evalu¬ 
ating  and  diagnosing  student  performance,  provid¬ 
ing  remedial  practice)  that  the  instructors  feel 
pride  and  security  in  being  uniquely  qualified  to 
perform  can  be  procedural i zed,  automated  and/or 
made  more  objective.  In  these  ways,  RAD  can 
threaten  the  status  of  user  as  instructors  (i.e., 
instructional  technology  might  replace  them  or 

at  least  diminish  their  importance  —  the  "John 
Henry  effect"). 

5.  The  use  of  low-cost,  low-fidelity 
trainers,  although  able  to  provide  effective 
training,  can  be  interpreted  by  Instructors  as 
another  sign  that  they  are  "less  elite"  than 
other  occupational  specialties  in  that  their 
group  has  to  "make  do"  with  "inferior  supplies". 

6.  Instructor  time  spent  in  a  trainer  can 
decrease  instructor  time  with  operational  train¬ 
ing  equipment  (e.g.,  aircraft).  However,  career 
advancement,  as  well  as  personal  enjoyment,  often 
seems  to  depend  on  use  of  operational  (vs.  simu¬ 
lated)  equipment.  Thus,  RAD  that  seeks  to  re¬ 
duce  the  inefficiencies  of  training  with  opera¬ 
tional  equipment  can  threaten  both  the  enjoyment 
and  career  advancement  of  the  user. 

7.  Although  the  principle  of  feedback  is 
central  to  both  motivational  and  learning  theory, 
RAD  personnel  in  training  have  been  criticized 
for  not  providing  knowledge  of  RAD  results  to  the 
participating  training  organization.  In  a  recent 
example,  an  operational  squadron  was  less  than 
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enthusiastic  about  starting  a  new  study  partly 
because  they  hadn't  received  even  the  final  re¬ 
port  of  a  study  completed  two  years  earlier. 

8.  There  are  predictable  psychological  re¬ 
sponses  to  change  which  lead  Instructors  (and 
others)  to  resist  Innovation.  Such  resistance 
Is  well  documented  (33)  under  a  variety  of  condi¬ 
tions,  including  Innovations  In  training  and 
educational  methods: 

a.  Basic  security  needs  will  be  com¬ 
promised  in  the  face  of  uncertainty  induced 
by  the  changes.  For  example,  the  instructor 
can  feel  unqualified  to  Implement  new  proce¬ 
dures. 

b.  A  second  natural  response  to  the 
innovation  is  that  more  effort  will  be 
necessary  to  implement  the  change  than  that 
required  to  continue  older  procedures. 

c.  It  is  also  frequent  for  the  imple¬ 
menting  organizational  unit  to  feel  criti¬ 
cized  In  the  face  of  the  change"!  No  amount 
of  logical  persuasion  and  discussions  re¬ 
garding  the  value  of  change  can  completely 
counteract  the  instructor's  conclusion  that 
current  performance  may  not  be  adequate. 

Since  there  are  few  absolute  standards  of 
the  value  of  his  work,  the  work  of  the  imple¬ 
menting  instructor  personnel  is  judged  large¬ 
ly  on  the  basis  of  relative  standards,  in¬ 
cluding  R&D  evaluations. 

d.  Finally,  the  members  of  the  imple¬ 
menting  organization  will  likely  experience 
some  degree  of  loss  of  freedom  (or  even  per¬ 
ceived  manipulation)  by  an  outside  agent. 

In  discussing  the  personal  response  to  man¬ 
agerial  control  systems.  Cl el and  and  King 
(9,  pp.  330-331)  have  identified  several 
common  yet  unintended  consequences  of  con¬ 
trol  systems  (as  paraphrased  from  McGregor, 
21).  Managerial  control  and  related  re¬ 
strictions  on  freedom  initiated  by  R&D  may 
yield:  (1)  failure  to  comply  with  the  full 
requirements  of  the  control  system;  (2) 
antagonism  to  the  controls  and  to  those  who 
administer  them;  (3)  unreliable  performance 
Information  due  to  erroneous  and/or  mislead¬ 
ing  reporting;  (4)  necessity  for  close  sur¬ 
veillance  by  managerial  staff;  and  (5) 
high  adnini strati ve  costs  due  to  the  expense 
of  surveillance. 

Deficiencies  in  User  Role  Assignments.  Inade- 
quate  liaison  between  R&D  personnel  and  users 
also  exists  because  of  a  lack  of  proper  defini¬ 
tion,  acceptance  and  performance  of  social  roles 
for  various  members  of  an  R&D  project.  The  social 
role  literature,  usually  defines  two  kinds  of 
role  "stresses"  —  role  ambiguity  and  role  con¬ 
flict.  Role  ambiguity  refers  to  a  lack  of  clear 
definition  of  one's  appropriate  role  behaviors; 
role  conflict  refers  to  two  or  more  clearly  de¬ 
fined  roles  In  which  simultaneous  performance 
prevents  either  role  holder  from  successfully 
fulfilling  his  responsibilities.  A  prevalent 
example  of  role  conflict  Is  between  the  R&D 
scientific  role  responsibility  and  the  Instruc¬ 
tor's  primary  role  to  rapidly  provide  trained 
pilots,  which  very  often  requires  training  com¬ 
mand  resources  completely  devoted  to  the  "ur¬ 


gency"  of  meeting  the  squadron  training  needs. 

To  further  aggravate  the  situation,  many  R&D  ef¬ 
forts  can  provide  only  partial  answers  to  train¬ 
ing  problems,  which  are  not  solved  until  several 
such  answers  are  pieced  together.  The  coopera¬ 
tion  of  users  can  become  severely  strained  when  it 
becomes  clear  that  the  slow,  cautious,  methodical, 
self-critical  approach  of  the  R&D  project  is  not 
going  to  provide  answers  fast  and  sure  enough  to 
satisfy  their  current  operational  needs.  Opera¬ 
tional  restrictions  on  experimental  control  can 
slow  the  R&D  process  by  necessitating  further  R&D 
to  answer  questions  about  possible  influences  of 
uncontrolled  variables. 

A  deficiency  more  serious  than  that  of  role  con¬ 
flict  is  role  ambiguity.  There  Is  no  basic  con¬ 
flict  between  users  and  R&D  personnel  in  terms 
of  their  basic  responsibilities  for  enhancing 
training  effectiveness.  There  Is  a  problem,  and 
resultant  stress  for  both  parties,  however,  when 
roles  are  ambiguous,  as  currently  exist  for  users 
working  with  R&D  personnel.  Clear  roles  defining 
expected  liaison  behaviors  are  absent. 

There  are  at  least  two  predominant  kinds  of  response 
to  role  ambiguity.  One  may  become  "irresponsible" 
and  give  up  trying  to  play  any  role  at  all.  Or, 
an  individual  may  try  to  assume  all  the  behav'ors 
that  might  fall  under  the  jurisdiction  of  his 
usual  organizational  responsibil ities.  This 
latter  response  is  fairly  typical  of  pilot  train¬ 
ing  in  which  Instructors  usually  approach  their 
job  with  a  high  sense  of  responsibility.  How¬ 
ever,  role  assignments  are  inappropriate  when 
users  attempt  to  evaluate  technical  research  as¬ 
pects  of  R&D  efforts,  such  as  research  design 
and  statistical  analysis.  When  users  begin 
taking  on  the  role  of  R&D  personnel,  a  third  kind 
of  role  stress,  which  could  be  called  "role  over¬ 
lap",  develops.  With  role  overlap,  neither  party 
in  the  relationship  can  feel  responsible  for 
successful  role  performance  since  there  are  mul¬ 
tiple  claims  on  "ownership"  of  the  role  behaviors. 
Like  role  ambiguity,  role  overlap  can  also  lead 
to  irresponsibility.  If  skilled  R&D  personnel 
are  not  given  the  opportunity  by  users  to  deal 
with  the  technical  research  Issues,  they  are  not 
"response-able";  if  highly  influential  users  do 
not  have  the  necessary  research  expertise,  they 
are  not  "able-to-respond". 

There  are  several  areas  in  which  inappropriate  role 
overlap  currently  exists: 

1.  Replication  of  a  previous  study  (where¬ 
by  the  experimental  conditions  of  an  earlier 
experiment  are  essentially  reproduced)  often  is 
viewed  by  users  as  "duplication  of  effort".  Du¬ 
plication  of  effort  occurs  and  should  be  avoided, 
according  to  many  users,  where  two  or  more  R&D 
projects  address  the  same  general  goals  (e.g., 
optimization  of  the  same  training  device).  How¬ 
ever,  when  the  R&D  objectives  have  critical  im¬ 
plications  for  hunan  lives,  money,  and  mission 
success,  multiple  approaches  to  the  same  R&D 
objectives  can  be  quite  desirable  and  even  imper¬ 
ative.  Equally  important,  replication  plays  a 
critical  scientific  role  In  the  R&D  process  in 
terms  of  defining  the  external  validity  or 
general Izablllty  of  previous  findings  across 
time,  subjects,  assumed  irrelevant  variations  In 
experimental  procedures,  etc.  In  the  case  of 
truly  independent  replications,  R&D  findings  be- 
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come  extremely  credible  where  similar  conclusions 
are  reached.  Behavioral  research  is  replete  with 
experimental  findings  that  cannot  be  replicated 
even  under  the  best  of  conditions.  Thus,  repli¬ 
cation  needs  to  play  a  much  greater  role,  espe¬ 
cially  in  operational  settings  of  minimal  exper¬ 
imental  control . 

2.  A  relatively  large  sample  size  is  con¬ 
sidered  by  the  user  to  be  essential  in  order  for 
any  objective  of  the  R&D  project  to  be  met.  How¬ 
ever,  the  size  of  an  adequate  experimental  sam¬ 
ple  is  a  complex  matter  involving  consideration 
of  a  nunber  of  design  and  statistical  issues 
(e.g.,  the  use  of  repeated  measures,  the  error 
variance  of  the  sample  population,  the  extent  to 
which  experimental  controls  can  be  implemented), 
and  the  nature  of  the  study  objectives. 

3.  Operational  users  often  want  to  consi¬ 
der  the  cost  of  the  R&D  as  part  of  their  evalua¬ 
tion  of  the  project.  This  practice  is  inappro¬ 
priate  because  R&D  costs  should  be  evaluated  in 
a  context  of  its  contribution  to  technological 
information  bases,  as  well  as  to  the  development 
of  specific  new  products.  Since,  at  best,  users 
only  have  information  related  to  the  value  of 
potential  new  products  for  their  particular 
situation,  user  efforts  to  evaluate  the  general 
costs  and  benefits  of  the  R&D  are  not  completely 
meaningful . 

4.  The  need  for  R&D  is  diminished  in  the 
view  of  users  to  the  extent  that  they  perceive 
their  current  training  program  as  already  success¬ 
ful.  However,  a  major  role  activity  of  R&D 
scientists  is  to  provide  an  independent  assess¬ 
ment  of  organizational  effectiveness.  In  addi¬ 
tion,  their  responsbility  is  to  help  assure  that 

a  successful  program  is  preserved  over  time 
through  documentation  and  standardization  of 
training  procedures.  Further,  it  is  important 
to  determine  as  precisely  and  accurately  as  pos¬ 
sible  what  the  criteria  for  training  program  suc¬ 
cess  are,  how  well  the  training  is  in  fact  work¬ 
ing,  and  what  specific  aspects  of  the  training 
are  contributing  to  or  detracting  from  the  level 
of  success  observed. 

5.  It  is  often  expected  by  users  that  de¬ 
tails  of  the  research  design  for  solving  the  R&D 
problem  will  be  discussed  veiy  early  in  the  pro¬ 
ject  and  often  in  the  first  kick-off  meeting  with 
users.  This  view  conflicts  with  R&D  requirements 
to  delay  specification  of  such  details  until 
appropriate  rationale  for  such  details  can  be 
obtained,  which  often  involves  the  cooperation 

of  the  users  who  are  requesting  the  details.  It 
often  happens  that  the  R&D  team  is  pressured  in¬ 
to  becoming  too  detailed  too  soon,  with  the  re¬ 
sult  that  the  R&D  design  is  quickly  criticized 
as  being  Inappropriate  for  the  particular  opera¬ 
tional  situation  in  question.  Such  criticisms 
are  usually  valid  since  a  major  contribution  of 
users  to  R&D  is  to  help  determine  an  R&D  design 
that  is  well  suited  to  their  particular  situation. 
Thus,  discussions  at  initial  meetings  between 
R&D  personnel  and  users  should  include  only  the 
level  of  detail  needed  to  evaluate  the  general 
R&D  approach  in  terms  of  specific  operational 
constraints  such  as  current  training  schedule. 
Instructor  personnel  and  student  throughput. 

R&D  managers  have  often  been  deficient  in  the 


application  of  participative  management  (PM)  to 
capitalize  on  the  user's  expertise.  PM  could 
assist  to  more  clearly  define  appropriate  user 
roles  (see  below). 

Deficiencies  in  Official  Navy  Organizational 
Policy  and  Structure.  There  are  a  number  of  con¬ 
straints  on  acceptance  which  have  little  or  noth¬ 
ing  to  do  with  Individual  motivation  of  the  user 
or  his  social  role  interactions  with  R&D  per¬ 
sonnel  .  Direction  from  official  policy  at  the 
training  organization  or  even  regional  or  nation¬ 
al  level  of  influence  can  restrict  the  success¬ 
ful  introduction  of  R&D.  An  understanding  of 
some  of  these  policies  can  be  extremely  valuable 
because  policy  changes  can  help  to  overcome  "Navy 
Policy"  obstacles.  Less  directly,  pol icy  modifi¬ 
cations  also  can  aid  in  resolving  the  constraints 
discussed  throughout  this  paper. 

1.  There  is  a  prevalent  and  generally  sup¬ 
ported  view  in  the  Navy  that  R&D  should  be  con¬ 
ducted  with  no  interference  with  ongoing  opera¬ 
tional  training.  Semple  (29),  e.g.,  has  specifi¬ 
cally  advocated  a  guest-host  relationship  in 
conducting  training  effectiveness  evaluations 
with  operational  user  hosts.  A  policy  statement 
from  the  Chief  of  Naval  Education  and  Training 
(CNET)  subsequently  formalized  Semple's  not-to- 
interfere-with-training  concept.  However,  since 
most  important  R&D  conducted  in  operational  set¬ 
tings  requires  some  such  "interference",  this 
conception  often  is  counter-productive.  Lacking 
a  generally  accepted  definition  of  what  inter¬ 
ference  means  and  how  it  applies  to  various  R&D 
situations,  this  view  can  be  taken  literally  and 
enforced  by  the  user  in  cases  where  it  should 
not  be.  A  policy  of  non-interference  also  con¬ 
tributes  to  the  impression  that  R&D  is  a  low 
priority  activity  which  is  to  be  tolerated,  if 
necessary,  but  is  not  to  be  given  serious  consi¬ 
deration  by  the  user  organization. 

2.  Current  Navy  policy  requires  frequent 
instructor  rotation  and  training  support  person¬ 
nel  turnover.  Thus,  in  the  process  of  introduc¬ 
ing  an  R&D  project,  one  or  more  key  personnel 

are  almost  always  about  to  leave  the  organization. 
For  these  individuals  there  Is  no  incentive  to 
make  a  commitment  to  the  project  since  any  bene¬ 
fits  resulting  from  the  project  probably  will  not 
occur  until  long  a^ter  their  rotation  date.  In 
addition,  all  the  risks  and  uncertainty  of  change 
cotimon  to  the  initiation  of  any  R&D  project  will 
be  experienced  if  a  consul  tment  is  made.  Further¬ 
more,  frequent  personnel  rotations  create  prob¬ 
lems  when  new  users  come  to  the  project  subsequent 
to  its  initiation.  Efforts  to  indoctrinate  users 
to  the  project  need  to  be  repeated  each  time  a 
new  member  arrives.  The  new  project  members  do 
not  have  the  benefit  of  feeling  that  their  efforts 
have  Influenced  the  determination  of  project  goals 
or  the  methods  used  to  achieve  them.  The  issue 
of  turnover  is  a  crucial  one  In  gaining  initial 
acceptance  as  well  as  continuity  of  acceptance 
throughout  the  project. 

3.  The  typical  aviation  training  organiza¬ 
tion  does  not  provide  guidance  for  adequate  in¬ 
structor  pilot  (IP)  training.  Current  instructor 
training  policy  is  almost  entirely  on  the  job 
training  (OJT)  conducted  very  informally  and 
sporadically  as  primary  training  schedules  and 
other  consul tments  permit.  The  training  provided 
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to  prospective  instructors  is  not  a  basic  but  a 
collateral  duty  of  the  "qualified"  instructor. 

As  a  consequence  of  inadequate  instructor  train¬ 
ing  to  operate  ATDs,  RAD  with  ATDs  is  less  like¬ 
ly  to  be  accepted  for  at  least  three  major  rea¬ 
sons: 

a.  The  new  instructor  is  forced  to 
develop  his  own  unique  training  tech¬ 
niques  used  with  the  ATD.  The  emotional 
commitment  to  methods  that  one  has  developed 
himself  is  generally  much  stronger  than 
those  recommended  or  required  by  other  peo¬ 
ple,  such  as  RAD  personnel  (cf.,  32).  Thus, 
there  is  likely  to  be  much  greater  resis¬ 
tance  to  a  proposed  change  in  training  tech¬ 
niques  than  if  the  techniques  were  relative¬ 
ly  standardized  across  instructors. 

b.  The  lack  of  standardized  instruc¬ 
tor  training  and  consequent  diversity  of 
opinion  about  what  is  important  in  training 
make  it  very  difficult  to  achieve  a  cohesive 
user  position  in  support  of  certain  proposed 
changes  in  ATD  training.  In  addition,  the 
integration  of  new  standardized  procedures 
with  instructors  who  previously  have  used 
highly  heterogeneous  training  procedures  is 

a  problem  because  the  relationship  of  the  new 
procedures  to  existing  ones  needs  to  be 
examined  on  an  individual  instructor  basis. 

c.  As  device  utilization  methods  pass 
informally  from  older  to  newer  instructors, 
much  of  the  original  information  is  lost 
due  to  limitations  in  recall,  oversimplifi¬ 
cation,  etc.  The  natural  oversimplification 
of  the  capabilities  of  ATDs  over  time  helps 
explain  the  well  documented  fact  of  under¬ 
utilization  (e.g.,  using  an  OFT  as  a  CPT) 

of  a  wide  variety  of  Air  Force  and  Navy 
ATDs.  Thus,  research  projects  which  attempt 
to  foster  the  use  of  the  full  capabilities 
of  an  ATD  which  has  been  passed  on  by  sever¬ 
al  generations  of  instructors  may  meet  with 
considerable  resistance  from  current  in¬ 
structors. 

4.  With  certain  exceptions,  such  as  the 
presence  of  Fleet  Project  Team  (FPTs),  local 
training  commands  often  do  not  have  policy  sup¬ 
port  for  the  personnel  and  resources  required  to 
accoimodate  an  RAD  effort.  Some  organizations 
are  genuinely  overloaded  with  existing  opera¬ 
tional  training  responsibilities. 

Inadequate  Defense  RAD  Contracting  Methods. 
Government  contracts  involving  operational 
training  organizations  tend  to  Ignore  the  real¬ 
ity  of  the  user  acceptance  problem  until  the 
contract  work  has  begun.  Scientific  officers 
monitoring  these  contracts  seldom  require  the 
contractor  to  develop  a  methodology  for  gaining 
acceptance.  Nor  is  there  a  contractual  require¬ 
ment  to  document  successful  and  unsuccessful 
acceptance  methods.  Consequently,  there  are  no 
contractual  provisions  for  the  necessary  time 
in  the  early  part  of  the  schedule  to  achieve 
acceptance  milestones  nor  are  there  funds  al¬ 
lowed  to  achieve  and  to  document  these  milestones. 
Thus,  each  new  RAD  study  is  forced  to  "rush 
through"  the  acceptance  problems  by  placing 
further  pressure  on  the  user.  This  pressure  is 
applied  without  benefit  of  the  lessons  learned 


from  previous  studies  that  have  experienced 
similar  difficulties. 

Inadequate  Integration  of  the  User  Into  The 
trainer  System  Acquisition  Process:  A  Lack  of 
Participative  fenagement.  The  formal  structure 
of  the  Weapon  System  Acquisition  Process,  includ¬ 
ing  training  system  development,  requires  user 
input  and  participation  at  several  points  in  the 
process  of  ATD  acquisition.  Caro  et  al.  (7) 
have  found  that  participation  of  Air  Force  users 
during  ATD  design  as  well  as  during  initial  ATD 
effectiveness  evaluation  in  the  training  envir¬ 
onment,  significantly  Improves  attitudes  toward 
the  ATD.  Mecherikoff  and  Hackle  (22)  found  that 
a  lack  of  such  participation  among  Navy  person¬ 
nel  created  a  strong  negative  attitude  of  "not 
invented  here."  There  generally  has  been  little 
systematic  application  of  participative  management 
(PM)  techniques  to  the  entire  process  of  trainer 
acquisition  and  subsequent  evaluation.  Including 
initial  effectiveness  evaluations  and  later  ATD 
training  optimization  studies.  PH  has  not  been 
applied  due  largely  to  the  three  limitations 
indicated  below: 

1.  Lack  of  recognition  of  the  value  of  PM 
as  a  general  tool  for  gaining  acceptance  of 
change. 

2.  Lack  of  operational  definitions  as  to 
how  PM  should  be  Implemented  during  a  specific 
proposed  change  in  training  methods. 

3.  Lack  of  formal  documentation  of  success¬ 
ful  applications  of  PM  to  specific  military 
training  organizations. 

A  successful  Introduction  of  an  R&D  study  will 
be  more  likely  if  it  is  made  clear  to  users  as 
soon  as  possible  that  a  participative  approach  is 
desired.  There  is  considerable  empirical  support 
for  the  value  of  PM  as  a  general  tool  for  intro¬ 
ducing  organizational  change  (16).  In  addition 
to  the  empirical  basis  for  using  PM,  there  are  a 
number  of  more  "intuitive/logical"  reasons  for 
using  PM: 

1.  PM  makes  use  of  subject  matter  experts 
(SMEs)  from  the  training  command  to  provide 
necessary  information  to  RAD  personnel.  Informa¬ 
tion  regarding  current  training  system  practices 
and  resources  as  well  as  suggestions  for  alterna¬ 
tive  RAD  approaches  and  methods  are  especially 
critical  in  the  complex  system  in  which  aerospace 
training  takes  place. 

2.  PM  allows  the  SME  to  1 earn  that  he  is 
valuable.  A  natural  response  to  change,  de¬ 
scribed  earlier,  is  a  feeling  that  one  is  being 
criticized  for  inadequacies.  With  PM  used  to 
introduce  change,  the  SME  learns  that  he  is  not 
being  replaced.  Certain  behavioral  practices  of 
training  may  be  replaced,  however.  To  the  ex¬ 
tent  that  SMEs  feel  that  they  are  being  evalu¬ 
ated,  PM  encourages  them  to  participate  In  deter¬ 
mining  the  basis  on  which  their  efforts  can  be 
judged  (16). 

3.  PM  reduces  the  natural  resistance  due 
to  uncertainty  of  an  unknown  change  and  its 
consequences. 

4.  PM  reduces  the  natural  resistance  due 
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to  the  perceived  loss  of  freedom  initially  attri¬ 
buted  to  presence  of  R&D  personnel .  The  PM  ap¬ 
proach  encourages  full  expression  of  dissenting 
opinions  and  is  consistent  with  the  democratic 
decision-making  ideals  generally  held  by  this 
country. 

5.  PM  allows  the  user  to  understand  the 
likely  level  of  complexity  of  the  proposed  change. 
If  the  user  does  not  grasp  how  everything  ties 
together  in  the  R&D  plan,  he  is  likely  to  make 
changes  (especially  an  upgrading  of  requirements) 
during  the  study  without  evaluation  of  the  proba¬ 
ble  impact  of  changes  on  technical  performance, 
schedule,  or  cost  risks  (10). 

6.  PM  provides  a  context  for  group  activi¬ 
ty  which  can  satisfy  certain  social  needs  such 

as  affiliation. 

7.  PM  identifies  personal  sources  of 
necessary  support  for  the  change  who  were  not 
initially  part  of  the  group  participation. 

8.  PM  informs  “outsiders"  of  the  informal 
rules  of  the  organization  which  can  complement 
the  formal  support  of  change. 

9.  As  a  result  of  a  number  of  the  forego¬ 
ing  advantages  of  PM,  PM  increases  the  organiza¬ 
tional  members'  commitment  to  the  change  goals 
and  objectives  established.  This  consequence  of 
PM  is  the  single  most  valuable  benefit  of  PM  and 
also  has  been  empirically  documented  (16). 
Participating  in  the  development  of  a  change  ef¬ 
fort  gives  one  the  experience  of  "owning"  the 
change  and  responsibility  for  supporting  its 
implementation.  Thus,  the  Implemented  change  is 
much  more  likely  to  be  long-lasting,  perhaps 
even  eventually  institutionalized  into  the  train¬ 
ing  organization. 

The  value  of  PM  as  a  general  tool  for  gaining 
acceptance  of  change  is  becoming  more  widely  ac¬ 
knowledged.  However,  there  are  few  operational 
definitions  of  how  to  implement  PM  into  opera¬ 
tional  military  systems.  However,  there  is  at 
least  one  significant,  but  preliminary  attempt  to 
proceduralize  PM  into  the  evaluation  of  complex 
aviation  systems  throughout  the  weapon  acquisi¬ 
tion  process.  Butterbaugh,  Moss,  Sexton,  and 
Kearns  (5)  have  developed  an  acquisition  process 
model  for  the  Air  Force  that  focuses  extensively 
on  user  input  evaluations  throughout  the  process. 

It  should  not  be  assumed  that  PM  is  a  panacea 
for  all  organizational  change  goals.  The  success 
of  PM  over  the  alternative  change  strategy  of 
directive  management  (DM)  depends  on  the  results 
of  organizational  diagnosis  by  the  change  advo¬ 
cates.  This  overall  diagnosis  would  not  only 
tell  the  R&D  personnel  whether  change  should  be 
attempted  at  all,  but  also  which  change  strategy 
would  be  more  likely  to  be  successful  with  the 
particular  organization  involved.  The  prime 
factor  determining  the  relative  efficacy  of  PM 
and  DM,  according  to  research  in  situational 
leadership  theory  (16),  is  the  level  of  "task¬ 
relevant  maturity"  diagnosed  In  the  group  members. 
People  with  high  task  maturity  are  "achievement- 
oriented,  seek  responsibility,  and  have  a  degree 
of  knowledge  and  experience  that  may  be  useful 
in  developing  new  ways  of  operating  ...  A  direc¬ 
tive  change  style  is  inconsistent  with  their  per¬ 


ceptions  of  themselves  as  mature,  responsible, 
self-motivated  people  who  should  be  consulted 
throughout  the  change  process",  (p.  283).  This 
description  of  high  maturity  seems  to  fit  very 
closely  the  operational  users  of  primary  interest 
in  this  report  -  namely  pilots.  An  earlier  sec¬ 
tion  of  this  report  pointed  out  that  the  typical 
instructor  pilot  has  an  intense  responsibility 
to  his  operational  tasks.  Such  responsibility, 
combined  with  the  usual  high  degree  of  task-rele¬ 
vant  expertise  among  pilots,  is  an  extremely 
valuable  asset  in  improving  training  systems. 

Other-Then-Rational  User  Responses  to  R&D  Studies 
In  training-!  Different  methods  of  introducing 
R&D  into  operational  military  training  organiza¬ 
tions  can  be  used.  At  a  heuristic  level,  there 
are  three  general  methods  used  to  gain  acceptance 
of  R&D.  The  user  may  be  told  that  he:  (1)  shall 
do  it  (power  of  higher  authority);  or  that  he 
(2)  can  do  it  (power  of  persuasion  based  on 
rational/logical  arguments  supporting  the  techni¬ 
cal  feasibility  of  a  proposed  change);  or  that 
he  (3)  wants  to  do  it  (power  based  on  the  user's 
own  emotional /motivational /value  system  which  is 
consistent  with  or  satisfied  by  the  proposed  R&D 
study).  Ideally,  all  three  reasons  exist  to  sup¬ 
port  acceptance.  Seldom,  however,  is  there 
clear  and  powerful  "shall  do"  authority  for  ini¬ 
tiating  specific  R&D  studies;  R&D  personnel  are 
normally  staff  officers  without  conmand  authori¬ 
ty. 

The  typically  used  method  Involves  rational/ 
logical  appeals  to  the  operational  users'  intel¬ 
lect  based  on  the  training  value  of  the  proposed 
R&D  effort.  This  "rationalistic  bias"  of  R&D 
specialists  is  also  found  in  non-military  organ¬ 
izational  change  efforts  (33)  and  assumes  that 
change  requires  no  more  than  an  exchange  of 
technical  knowledge.  One  of  the  major  constraints 
on  gaining  user  acceptance,  however,  is  that  ac¬ 
ceptance  of  change  is  naturally  Influenced  by 
emotional  factors  that  are  not  strictly  "ration¬ 
al"  in  nature.  In  the  Navy  maintenance  training 
area,  e.g.,  there  exist  severe  acceptance  prob¬ 
lems  of  training  devices  although  these  devices 
were  designed  and  validated  according  to  princi¬ 
ples  of  modularity,  self-pacing,  criterion-ref¬ 
erencing,  etc.  Even  where  technical  arguments 
are  clearly  made,  they  are  not  always  sufficient. 
It  is  argued,  therefore,  that  the  successful  R&D 
scientist  must  have  skills  which  allow  him  to 
anticipate  and  deal  with  these  “other-than- 
rational"  user  responses  which  can  lead  to  lack 
of  acceptance  of  a  proposed  study. 

These  responses  to  the  Introduction  of  an  R&D 
study  occur  In  many  forms,  but  they  can  be  clas¬ 
sified  in  at  least  three  basic  ways.  Some  exam¬ 
ples  from  naval  aviation,  as  observed  by  various 
scientific  officers  and  contractors  working  with 
the  Naval  Training  Equipment  Center  is  provided 
for  each  of  these  classifications.  (A  follow-on 
paper  will  attempt  to  describe  these  same  kinds 
of  responses  by  scientific  officers,  as  observed 
by  operational  users). 

1.  Emotional  resistance  based  on  the  part¬ 
icular  operational  organization's  situation. 
Comnonly  heard  statements  by  users  Include: 

a.  "Our  situation  is  unique  -  train¬ 
ing  results  found  elsewhere  don't  apply  to 
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us", 

b.  "Our  situation  Is  too  complex  to 
study". 

c.  "Even  If  your  proposed  study  could 
Improve  our  training  for  now,  our  situation 
will  change". 

2.  Emotional  resistance  based  on  the  lack 
of  credibility  of  the  R&D  specialist! 

a.  "You  can't  evaluate  the  flight 
simulator  If  you  haven't  flown  the  actual 
aircraft". 

More  generally,  this  response  Implies  that  opera¬ 
tional  experience  Is  a  prerequisite  for  credibility 
as  an  evaluator.  This  view  would  be  perfectly 
rational  if  the  evaluator's  responsibility  was  to 
improve  or  practice  his  skills  as,  for  example, 
a  pilot.  Some  users  even  take  the  extreme  view 
that  evaluators  must  be  equally  proficient  at 
operating  the  actual  equipment.  Obviously,  how¬ 
ever,  the  user  is  the  operational  expert  and  it 
would  not  be  cost-effective  to  require  of  R&D 
specialists  more  than  a  minimal  degree  of  opera¬ 
tional  experience.  Logically,  the  R&D  scientist 
could  also  insist  that  the  user  be  as  skilled  as 
he  at  performing  R&D.  Of  course,  this  logical 
alternative  to  requiring  the  R&D  specialist  to  be 
operationally  experienced  is  seldom  stated  since 
it  is  recognized  that  there  are  necessary  and 
complementary  forms  of  expertise  represented  by 
both  operational  and  R&D  personnel .  If  the  R&D 
specialist's  skills  overlapped  completely  with 
those  of  the  user  he  would  have  nothing  unique  to 
contribute  to  improving  training  effectiveness. 

Although  it  would  be  difficult  to  insist  that 
operational  experience  is  detrimental  to  one’s 
effectiveness  as  an  evaluator,  it  Is  not  always 
possible  to  show  that  such  experience  Improves 
effectiveness.  It  has  long  been  thought  In  clin¬ 
ical  psychology,  e.g.,  that  direct  experience 
with  a  client's  symptoms  would  facilitate  a 
counselor's  effectiveness.  However,  specific 
research  (30)  with  alcoholics  treated  by  previous 
alcoholic  counselors  showed  that  success  rates 
were  no  higher  than  for  those  treated  by  "non- 
empathetlc"  counselors.  It  Is  probably  true  that 
such  empathy  would  increase  the  chance  of  the 
client  accepting  a  treatment  program  In  the  same 
sense  that  the  study  proposed  by  a  credible  R&D 
evaluator  would  be  accepted.  This  initial  accep¬ 
tance,  In  both  cases.  Is  based  more  on  emotional 
grounds  than  on  grounds  of  potential  effective¬ 
ness,  however.  The  user  can  easily  Identify  with 
and  accept  operational  task  experience  as  grounds 
for  credibility.  The  R&D  specialist's  expertise 
In  such  tasks  as  problem  definition,  development 
of  methodological  procedures,  statistical  analy¬ 
sis,  oral  and  written  communication  of  results, 
etc.,  none  of  which  are  dependent  on  operational 
experience.  Is  much  more  difficult  for  the  user 
to  appreciate. 

3.  Emotional  resistance  based  on  attitudes 
towards  ATDs  undergoing  evaluation. 

a.  "The  TDs  (Training  Device  Main¬ 
tenance  Personnel)  fly  the  simulator  better 
than  real  pilots". 


With  certain  carrier  landing  trainers,  users 
sometimes  use  TDs  to  provide  ATD  demonstrations 
to  visitors  because  "They  can  catch  the  #3  target 
wire  every  time".  Although  there  is  nothing 
Inherently  Irrational  In  this  statement  (it  is 
true),  the  underlying  attitude  Is  negative  towards 
the  ATD  since  It  is  Implied  that  the  simulator 
teaches  skills  not  required  by  an  experienced 
carrier  pilot.  The  llloglc  of  this  attitude  is 
based  on  acceptance  of  the  conclusion  that  TDs 
would  not  be  able  to  land  on  a  real  carrier  even  if 
they  were  proficient  In  the  simulator.  However, 

TDs  are  not  given  the  opportunity  to  actually 
attempt  carrier  landings  and  thus  there  Is  no 
empirical  basis  for  accepting  this  conclusion. 

Even  If  the  TD  could  not  land  on  the  real  carrier, 
presimtably  because  the  landing  trainer  does  not 
teach  all  the  skills  required  for  landing  success, 
a  significant  portion  of  the  required  skills 
learned  by  the  TD  would  transfer  to  the  actual 
landing  task. 

b.  "If  a  flight  simulator  doesn't 
feel  right,  it  has  no  training  value”. 

This  response  aptly  illustrates  a  general  pheno¬ 
menon,  observed  by  Mackle  et  al .  (20).  They 
found  that  users  often  feel  that  unless  the  ATD 
is  perfect  In  all  respects.  Its  use  for  any  reason 
is  regarded  as  a  waste  of  time.  The  device  is 
then  likely  to  be  regarded  as  a  "toy"  or  as  a 
"pinball  machine". 

Deficiencies  In  Training  Device  Design.  The  pri - 
mary  systematic  approach  to  acceptance  has  been 
identification  of  deficiencies  in  ATD  design 
(7,  20).  This  approach  Is  fundamental  to  the 
solution  of  acceptance  problems  because  the  de¬ 
ficiencies  can  be  fairly  easily  isolated  and 
future  ATD  design  presumably  modified  to  correct 
them.  Deficiencies  in  device  design  is  one  of 
the  most  severe  constraints  on  the  acceptance  of 
R&D  studies  Introduced  to  training  organizations. 
However,  It  Is  In  the  ATD  design  area  that  the 
scientific  approach  can  be  most  easily  applied 
to  predict  and  control  the  level  of  acceptance 
of  ATDs.  Changing  the  machine  (l.e.,  the  ATD) 
to  fit  the  man  (i.e.,  the  ATD  instructor  and 
student)  may  be  much  easier  than  attempting  to 
change  the  man  (user)  who  does  not  accept  the 
machine. 

The  acceptability  factors  in  ATD  design  fall  Into 
two  general  classes:  (1)  hardware/flrmware  fea¬ 
tures,  and  (2)  software/courseware  features. 

This  first  class  essentially  Includes  a  variety 
of  physical  fidelity  factors  relating  to  the 
student  work  station  (and  the  physical  layout  of 
the  Instructor  work  station).  It  would  Include, 
e.g.,  the  presence  and  appropriate  positioning 
(absolute  and  relative)  of  necessary  controls  and 
Information  displays  Inside  the  cockpit.  The 
second  class  Includes  both  physical  fidelity  fac¬ 
tors  created  by  computer  simulation  (e.g.,  visual 
and/or  motion  effects)  and  various  Instructional 
features  designed  Into  the  software. 

AN  INITIAL  MODEL  OF  THE  PROCESS  OF 

ACCEPTANCE  OF  R&D  -  INDUCED  CHANGE 

Figure  1  portrays  a  preliminary  model  of  accep¬ 
tance.  The  model  Identifies  these  stages  of  In¬ 
novation  as  formulated  by  K.  Lewln  (19)  and  the 
four  related  changes  conceptualized  by  Rogers 


411 


and  Rogers  (27).  The  first  stage  for  Lewin  is 
called  "unfreezing"  which  implies  that  change, 
e.g.,  introduction  of  an  R&D  study,  can  occur 
only  after  the  individual  or  organization  is 
ready  for  it.  Listed  under  this  stage  are  several 
kinds  of  factors  or  "forces"  (discussed  in  this 
paper)  that  would  decrease  the  risk  of  transition¬ 
ing  to  the  second  stage,  termed  "changing",  where¬ 
in  the  changes  would  actually  be  made. 

Separate  from  the  forces  that  would  support  change 
is  the  original  recognized  need  or  requirement  for 
change.  This  recognition,  of  a  discrepancy  be¬ 
tween  available  training  technology  and  current 
capabilities,  is  the  first  stage  of  innovation 
for  Rogers  and  Rogers.  Both  a  recognized  need 
for  change  and  the  presence  of  forces  supporting 


the  change  are  necessary  to  transition  to  the 
second  stage  in  which  changes  are  trial  tested, 
e.g.,  in  the  form  of  an  R&D  study.  The  last 
stage  for  both  theorists  is  "refreezing"  (19)  or 
institutionalization  of  the  change  (27),  e.g., 
integration  of  R&D  results  into  ongoing  training 
methods  to  improve  organizational  effectiveness. 

Five  levels  of  change,  ranging  from  a  change  in 
individual  knowledge  to  a  change  institutional¬ 
ized  in  organizational  policy,  adapted  from 
Hersey  and  Blanchard  (16),  is  also  presented  in 
Figure  1.  These  levels  of  change  correspond 
roughly  with  the  stages  of  the  innovation  pro¬ 
cess.  Knowledge  (e.g.,  of  the  latest  technology), 
the  lowest  level  of  R&D-induced  change,  is  the 
basis  for  Rogers  and  Rogers'  first  stage  in  which 
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FIGURE  1.  A  preliminary  model  of  the  acceptance  process.  Indicated  are  the  stages 
of  change  and  a  listing  of  factors  supporting  the  initial  stage  of  intro¬ 
ducing  an  R&D  study  (as  emphasized  in  this  paper).  The  use  of  personal 
power  typical  of  R&D  specialists  vs.  the  organizational  power  present  in 
the  training  command  structure  is  shown  applied  to  different  levels  of 
attempted  change.  The  model  includes  elements  from  Hersey  &  Blanchard 
(16),  Lewin  (19),  &  Rogers  &  Rogers  (27). 
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the  user  recognizes  the  existence  of  improved 
technology.  Their  second  stage,  which  is  the 
belief  or  feeling  that  change  is  necessary  or 
useful,  corresponds  to  the  next  highest  level  of 
change  in  which  attitudes  are  supportive  of  the 
change  proposed  by  the  R&D  specialist.  This  mo¬ 
tivation  to  change  is  the  third  necessary  ingre¬ 
dient,  along  with  recognition  of  a  need  and  the 
presence  of  driving  forces  supporting  change, 
for  successful  initiation  of  R&D. 

The  present  paper  has  focused  on  acceptance  of 
the  introduction  of  an  R&D  study.  The  stages  of 
innovation,  however,  also  include  individual  and / 
or  group  behavioral  commitments  in  order  to  trial 
test  the  changes  successfully  introduced  by  the 
R&D  study.  These  commitments  represent  more  ad¬ 
vanced  levels  of  acceptance  of  change  because 
they  include  all  the  lower  level  knowledge  and 
attitude  changes  and  are  therefore  more  diffi¬ 
cult  to  achieve.  The  highest  level  of  accep¬ 
tance  is  change  translated  into  institutional 
policy  such  that  it  is  no  longer  recognized  as  a 
change.  The  forces  supporting  changes  at  these 
higher  levels  of  acceptance  are  probably  similar 
to  those  driving  change  during  the  first  stage 
of  innovation  and  will  be  presented  in  other 
papers  in  this  area. 

The  model  also  indicates  that  the  "shall  do" 
kind  of  influence  based  on  the  power  of  position¬ 
al  authority  (13)  comes  from  the  top  organiza¬ 
tional  levels  such  as  the  commanding  officer  (CO) 
whereas  personal  power  starts  at  the  bottom 
levels.  R&D  personnel  are  ultimately  trying  to 
make  changes  at  a  relatively  high  level  (i.e., 
at  the  group  and/or  organizational  level).  If 
change  is  achieved  at  these  relatively  high 
levels,  the  change  is  more  permanent  and  may  even 
become  institutionalized.  However,  this  task  is 
extremely  difficult  since  R&D  personnel  are  usu¬ 
ally  limited  to  personal  power.  Whereas  a  CO 
could  initiate  an  immediate  and  dramatic  organi¬ 
zational  change  through  the  power  of  his  posi¬ 
tion,  the  R&D  specialist  must  begin  his  goal  of 
organizational  change  by  first  changing  knowledge 
of,  and  attitude  toward,  the  proposed  change. 

The  concept  of  personal  power  involves  the  power 
of  technical  content  expertise  in  interpersonal 
relations.  As  suggested  elsewhere,  most  human 
factors  R&D  personnel  have  not  fully  developed 
their  personal  power  of  influence  in  terms  of 
interpersonal  relations  based  on  an  understand¬ 
ing  of  the  operational  user's  emotional/motiva- 
tional/value  system  of  acceptance. 

ISSUES  IN  USER  ACCEPTANCE 

The  general  constraints  on  user  acceptance  appear 
to  be  a  useful  means  for  classifying  approaches 
to  acceptance  problems.  However,  there  are  a 
number  of  specific  issues  regarding  acceptance 
which  remain  to  be  addressed.  Two  of  the  more 
critical  and  controversial  issues  introduced  in 
this  paper  are:  (1)  how  are  subjective  user  re¬ 
ports  best  used  in  the  measurement  of  acceptance; 
(2)  to  what  extent  should  maximal  physical  fideli¬ 
ty  be  the  goal  of  training  system  design? 

These  two  issues  are  related  in  that  user  opinion 
often  seems  to  take  the  stance  that  higher  device 
fidelity  is  always  better.  As  indicated  earlier, 
physical  fidelity  seems  to  be  the  single  most 
Important  determinant  of  user  acceptability.  In 


contrast,  many  R&D  efforts  lead  to  the  conclusion 
that  deviations  from  maximal  physical  fidelity 
are  possible  and  even  desirable  in  the  interest 
of  cost-effective  training.  These  differences 
in  orientation  are  reflected  by  the  use  of  the 
concepts,  "simulator"  and  "trainer".  The  former 
term  has  come  into  vogue  with  unfortunate  conse¬ 
quences  on  training  efficiency,  since  it  implies 
that  training  devices  are  superior  only  when 
they  closely  resemble  operational  conditions. 

This  implication  is  strengthened  by  the  challenge 
from  users  that  training  designers  do  not  yet 
know  enough  about  training  to  risk  deviations 
from  maximum  possible  levels  of  simulation  fidel¬ 
ity.  However,  enough  is  known  now  to  deviate 
significantly  from  the  "simulation"  criteria  of 
acceptability  and  to  place  emphasis  on  the  de¬ 
vice's  "trainer"  features  (i.e.,  features  that 
facilitate  positive  transfer  to  operational  set¬ 
tings). 

Discussions  of  the  processes  by  which  training 
device  fidelity  (and  consequently  costs)  can  be 
reduced  and  learning  can  be  maintained  or  even 
increased  are  presented  in  a  variety  of  sources 
(e.g.,  4,  7,  18).  Two  separate  R&D  demonstra¬ 
tions  of  these  processes  are  underway  at 
NAVTRAEQUIPCEN.  A  low-cost,  low-fidelity  cockpit 
procedures  trainer  (CPT)  for  the  SH-3H  aircraft 
has  been  developed  and  partially  evaluated  and 
early  results  indicate  essentially  equivalent 
training  at  about  17  percent  of  the  cost 
($310,000  vs.  $1,800,000)  of  an  existing  more  con¬ 
ventional,  higher  fidelity  CPT  for  the  same 
aircraft.  Similar  approaches  are  being  taken 
and  similar  results  are  expected  with  a  low-cost 
part  task  trainer  for  the  EA-3B  aircraft. 

Further  evidence  corroborates  the  notion  that 
high  fidelity  is  not  essential  nor  even  necessar¬ 
ily  desirable  for  effective  learning  (18,  31). 

For  example,  up  to  50  percent  savings  in  flight 
hours  has  been  obtained  from  practice  on  low 
fidelity  flight  trainers.  Also,  fidelity  re¬ 
quirements  are  found  to  depend  on  factors  such 
as  the  amount  of  transfer  of  training  required, 
the  prior  experience  of  the  student,  and  strate¬ 
gies  of  instruction.  Numerous  studies  have 
demonstrated  that  the  training  of  aircraft  pilot¬ 
ing  can  be  effective  where  patterns  of  stick 
forces,  rather  than  the  absolute  amount  of  force 
or  displacement  of  stick  movement,  correspond 
with  operational  conditions.  Such  deviations 
from  high  fidelity  may  be  highly  justified  on  a 
cost-effective  basis,  but  rejected  by  users 
based  on  fidelity  considerations. 

Traditional  practice  has  been  and  to  a  large 
degree  still  is  to  buy  user  acceptance  by  in¬ 
creasing  the  fidelity  of  simulation.  A  topical 
question  regarding  this  practice  is:  how  much 
can  we  afford  to  spend  on  device  features  for  the 
sole  sake  of  user  acceptance?  A  trend  appears 
to  be  at  hand  where  such  expenditures  will  be 
minimized  and  more  low  cost  trainers  will  be  re¬ 
placing  high  cost  simulators. 

The  issue  remains,  however,  that  if  R&D  indicates 
the  desirability  of  lower  cost/fldelity  trainers, 
how  do  we  deal  with  subjective  impressions  of 
user  personnel  which  are  often  In  conflict  with 
their  use?  These  subjective  reports  are  the 
traditional  and  most  coronon  form  of  acceptance. 

One  approach  to  resolving  this  dilenma  is  to 
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empirically  ascertain  the  ability  of  users  to 
subjectively  assess  the  training  value  of  devices. 
This  can  be  done  in  part  by  determining  the  re¬ 
liability  of  user  opinion  (i.e.,  agreement  among 
users)  regarding  both  the  acceptability  and 
training  value  of  particular  device  features 
(17) .  According  to  Sammet  (28),  there  is  seldom 
a  cohesive  or  unitary  operational  user  when 
operational  needs  are  specified  by  the  user.  The 
same  lack  of  consistency  may  be  documented  for 
user  evaluations  of  ATOs.  However,  if  user 
opinion  is  reliable,  further  R&D  would  be  needed 
to  document  the  relationships  between  user  assess¬ 
ments  of  the  training  value  of  device  features 
and  actual  student  learning  with  these  features. 

In  this  way,  an  empirical  basis  for  the  utiliza¬ 
tion  of  user  evaluations  could  be  established. 
There  is  evidence  at  NAVTRAEQUIPCEN,  e.g.,  that 
user  judgments  of  the  necessity  of  motion  in 
flight  simulators  are  related  to  their  actual 
performance  in  these  devices  for  some  types  of 
motion  systems  but  are  not  for  others  (26). 

CONCLUSIONS 

The  user  acceptance  area  appears  to  be  in  a  con¬ 
dition  similar  to  the  instructional  development 
field  prior  to  the  advent  of  the  Instructional 
Systems  Development  (ISD)  era.  That  is,  no  sys¬ 
tematic  procedures  are  available  and  little 
sharing  of  information  occurs  regarding  naval 
user  acceptance  problems  and  their  solutions. 
Further,  very  little  science  has  developed  in  the 
area  and,  at  best,  practices  represent  a  very 
under-developed  art.  As  a  result,  one  must  de¬ 
pend  mainly  on  his  own  personal  experiences  and 
mistakes  and  his  own  independent  actions  to 
solve  the  problems. 

Given  the  obvious  cost  benefits  and  improved 
readiness  resulting  from  increased  acceptance, 
it  is  surprising  that  so  little  empirical  work 
has  been  done  to  identify  a  set  of  acceptability 
factors  and  factor  priorities  for  different 
classes  of  trainers.  Similarly  almost  no  guidance 
exists  as  to  how  to  successfully  introduce  R&D 
studies  into  training  organizations.  It  is  hoped 
that  this  paper  has  adequately  stated  the  sever¬ 
ity  and  scope  of  the  acceptance  problem.  A 
second  paper  is  planned  to  identify  solutions 
which  will  involve  overcoming  the  system  con¬ 
straints  on  acceptance  described  herein  and  to 
continue  development  of  the  model  of  the  accep¬ 
tance  process. 
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ABSTRACT 

The  classical  consideration  of  people  requirements  "after  the  fact"  in  weapons  systems  or 
simulator  design  instead  of  during  concept  definition  is  posing  ever  increasing  threats  to  the 
survival  of  new  programs.  Systems  in  a  military  environment  composed  of  manpower  shortages 
and  reduced  skill  levels  are  being  fielded  with  an  inadequate  provision  for  tailoring  equip¬ 
ment/software  which  minimize  skill  requirements  or  ease  of  performance  training.  People 
are  at  the  heart  of  any  system.  The  Personnel  Subsystem  is  one  of  the  major  system  compo¬ 
nents  which  must  be  considered  at  the  point  of  defining  the  Mission  Element  Needs  Statement 
(MENS).  Hardware  and  software  are  designed  to  extend  the  reach  of  the  available  manpower 
pool  to  meet  a  mission  need;  therefore,  0MB  A109  clearly  requires  that  the  definition  of 
the  personnel  subsystem,  including  the  training  and  training  equipment  requirements  to  develop 
the  personnel ,  must  commence  at  system  concept  formulation  and  continue  through  subsequent 
development  of  integrated  logistic  support. 


INTRODUCTION 

Considering  people  at  the  point  of  "MENS"  makes 
the  difference.  As  we  all  know,  personnel  con¬ 
stitute  a  key  part  of  all  systems.  The  objective 
of  this  presentation  is  to  discuss  the  importance 
of  considering  personnel  at  the  earliest  possible 
point  in  a  system  acquisition  process  for  major 
systems,  definition  and  outline  of  the  MENS,  and 
how  and  why  personnel  are  key  to  a  viable  MENS. 

Office  of  Management  and  Budget  (0MB)  Circular 
No.  A109  establishes  the  policies  to  be  followed 
by  executive  branch  agencies  in  the  acquisition 
of  major  systems.  The  circular  covers  procure¬ 
ment  of  all  major  systems  by  the  government.  In 


this  presentation,  only  DoD  system  acqusitions 
will  be  covered.  The  acquisition  process  is  cued 
by  the  mission  area  analysis.  See  Figure  1  - 
Acqusition  Process.  The  Secretary  of  Defense  in 
conjunction  with  the  DoD  Components  has  established 
mission  areas  essential  to  accomplish  the  Defense 
mission.  Each  DoD  Component  Head  has  established 
a  set  of  procedures  to  analyze  their  assigned  mis¬ 
sion  responsibilities.  This  continuing  analysis 
identifies  mission  elements  for  which  existing  or 
projected  capability  is  deficient  in  meeting  the 
essential  mission  needs  and  areas  where  existing 
capabilities  could  be  enhanced. 
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Figure  1  -  Acquisition  Process 
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When  the  Component  Head  identifies  a  deficiency, 
a  description  of  the  mission  need  is  submitted 
to  the  Secretary  of  Defense.  Once  this  Mission 
Element  Need  Statement  (MENS)  is  approved  by  the 
Secretary  of  Defense  the  acquisition  cycle  be¬ 
gins.  This  point  in  the  acquisition  cycle  is 
"Milestone  0".  This  portion  of  the  acquisition 
cycle  is  normally  unfunded  but  is  key  to  the 
acquisition  process. 

Recent  track  records  indicate  that  only  approxi¬ 
mately  252!  of  MENS  submitted  are  approved.  One 
of  the  major  rejection  criteria  has  been  that 
they  have  been  too  constraining.  The  Intent  of 
the  MENS  is  to  identify  a  need  not  its  solution. 
The  mission  need  may  best  be  satisfied  by  a 
change  in  doctrine,  by  deploying  additional  per¬ 
sonnel  or  equipment,  by  training,  by  trainers/ 
simulators,  by  modification  of  existing  equipment, 
or  by  a  new  major  system  acquisition;  however, 
the  selection  of  an  alternative  solution  is  the 
objective  of  the  next  phase  of  the  cycle- -not  the 
MENS.  The  MENS  must  state  only  the  need,  the 
current  capability  to  satisfy  that  need,  and  the 
constraints.  I  will  return  to  the  MENS  and  the 
importance  of  considering  personnel  a  little 
later. 

First,  I  would  like  to  proceed  with  an  overview 
of  the  acquisition  cycle  in  order  to  demonstrate 
the  relationship  of  the  MENS  to  the  total  cycle. 

ACQUISITION  CYCLE 

The  next  step  in  the  process  is  to  explore  and 
develop  systematically  and  progressively  the 
alternative  system  concepts  which  satisfy  the 
approved  need.  Once  this  is  accomplished  and  it 
i..  felt  that  selected  alternatives  should  be 
demonstrated,  the  DoD  Component  Head  requests 
approval  to  proceed  with  the  demonstration  and 
validation  of  the  alternatives.  The  request  is 
documented  in  a  set  of  recommendations  called  a 
Decision  Coordination  Paper  (DCP).  The  paper  is 
reviewed  by  the  Defense  System  Acqusition  Review 
Council  (DSARC)  and  the  (Service)  System  Acquisi¬ 
tion  Review  Council  { (S)SARC)  before  the  Secre¬ 
tary  of  Defense  makes  a  decision.  At  this  point, 
it  should  be  noted  that  MRA&L  (Office  of  the 
Assistant  Secretary  of  Defense  -  Manpower,  Reserve 
Affairs  and  Logsitics)now  has  a  seat  on  DSARC. 

This  assures  that  personnel  will  be  a  key  part 
of  the  total  process.  From  this  point  on 
assuming  appropriate  approvals,  the  system  goes 
thru  Milestone  II  -  Full  Scale  Engineering 
Development  and  Milestone  III  -  Production  and 
Deployment.  This  completes  the  acquisition 
cycle.  Let  us  now  go  back  to  the  MENS  phase 
and  further  explore  the  importance  of  personnel 
in  the  concept  definition  of  a  system. 


ASSUMPTIONS  OF  RELATIONSHIPS 

Some  assumptions  will  be  made  concerning  the 
human's  relationship  to  the  system. 

1.  There  is  a  relationship  between  the  effi¬ 
ciency  with  which  people  operate  within  a 
system  and  the  total  effectiveness  of  the 
system. 


2.  The  characteristics  of  the  system  and  t*  e 
hardware  within  the  system  influence  how 
men  operate  within  the  system, 

3.  It  is  cheaper  and  easier  to  adapt  equipment/ 
systems  to  human  capabilities  than  it  is  to 
modify  human  capabilities  to  system  require¬ 
ments. 

This  last  assumption  requires  some  explanation. 
Prior  to  World  War  II,  personnel  played  only  a 
small  part  in  the  total  system.  A  weapon  system 
consisted  of  a  fairly  simple  set  of  hardware  -  - 
operated  and  maintained  by  a  very  inexpensive, 
motivated  labor  force.  Today’s  weapon  system 
consists  of  ever-increasing  complex  hardware/ 
software  sub-systems  operated  and  maintained  by 
an  expensive,  sometimes  unmotivated  labor  force 
with  limited  prerequisites. 

MENS  CONTENT 

With  these  assumptions  in  mind,  let's  explore  the 
requirements  of  the  MENS.  An  outline  of  the  ele¬ 
ments  of  a  MENS  can  be  seen  in  Table  1  -  MENS  OUT¬ 
LINE. 

Table  1  MENS  OUTLINE 

1.  Mission 

2.  Threat  or  Basis  of  Need 

3.  Existing  and  planned  capabilities 
to  accomplish  the  mission  element 
need 

4.  Assessment 

5.  Constraints 

6.  Impact  of  staying  with  present 
capability 

7.  Resources  required  to  meet  Mile¬ 
stone  I 

The  first  element  of  the  MENS  is  a  statement  about 
the  mission  itself,  what  is  the  mission  area  and 
specific  subset(s)  of  the  mission  area. 

The  second  element  is  a  short  statement  on  the 
need, in  the  case  of  a  weapon  system  this  is 
usually  in  the  form  of  a  threat.  Quantification 
of  this  threat  or  need  in  numbers  and  capability, 
whenever  possible,  is  described. 

The  third  element  is  a  brief  summary  of  the 
existing  and  planned  DoD  and  allied  capability 
to  accomplish  the  mission. 

The  fourth  element  is  an  assessment  of  the  need 
expressed  in  one  or  more  of  the  following 
terms : 

1.  Deficiency  in  the  existing  capability 
in  areas  of  manpower,  performance, 
cost  effectiveness,  or  time  period  of 
the  existing  capability.  As  can  be 
seen,  people  can  induce  the  deficiency. 

2.  Technological  opportunity 

3.  Force  size 


4.  Physical  obsolescense  of  equipment 

5.  Cost  savings  opportunity,  life  cycle 
cost  potential  for  savings 

6.  Vulnerability  of  existing  systems 

The  fifth  element  is  the  constraints  placed 
upon  the  need  or  the  solutions.  These  include: 

1.  Affordability  in  relationship  to  the 
overall  Service  budget. 

2.  Relative  priority  within  the  mission 
area.  -  It  might  be  mentioned  here 
that  for  every  program  the  acquisi¬ 
tion  cycle  requires  a  yearly  assess¬ 
ment  of  the  relative  priority.  This 
is  what  causes  a  fluctuating  budget 
over  the  span  of  a  program.  It's 
priority  in  relation  to  other  programs 
on  a  yearly  basis  dictates  it's  yearly 
budget. 

3.  Logistics  and  manpower  consideration. 
Here  again  people  come  into  play.  A 
very  key  element  of  a  system  is  the 
constraint  put  on  the  system  by  it's 
personnel  and  logistics  support. 

4.  Standardization/conmonal ity  with  NATO 
as  well  as  among  the  other  Services. 

5.  The  timing  of  the  need. 

The  sixth  element  is  the  impact  of  staying  with 
the  present  capability.  This  includes  such 
things  as  ability  to  meet  the  threat  or  need, 
impact  on  combat  effectiveness,  and  the  cost 
of  product  improvements  to  the  current  system. 

The  seventh  element  is  the  identification  of 
resources  and  schedule  to  meet  Milestone  I. 

As  can  be  seen,  inputs  concerning  personnel  are 
required  in  several  elements  of  the  MENS. 
Specifically,  personnel  are  or  can  be  involved 
as  the  source  of  a  deficiency  or  need,  or  can 
constitute  a  constraint  to  the  solution  of  the 
need. 

A  little  later  on  in  the  presentation  I  will 
discuss  an  actual  MENS  and  give  examples  of  how 
some  of  these  elements,  especially  those  relat¬ 
ing  to  personnel  were  documented. 

REQUIREMENT  FOR  FRONT-END  PERSONNEL  ANALYSIS 

The  MENS  must  express  needs  and  program  objec¬ 
tives  in  mission  terms  and  not  equipment  terms 
or  solutions.  Innovation  and  competition  in 
creating,  exploring,  and  developing  alternative 
system  design  concepts  must  be  emphasized. 

Again  these  initial  activities  must  allow  for 
competitive  exploration.  As  stated  by  COR  Paul 
R.  Chatelier,  USN,  the  systems  of  the  future  must 
be  analyzed  and  supported  by  a  detailed  front-end 
analysis.  Key  issues  that  must  be  addressed  by 
DoD  Component  Heads  are: 

o  The  military  force  must  become  more 
effective  with  fewer  personnel 
o  Increased  competition  for  defense 
funds  between  weapons  and  manpower 
o  Personnel  must  become  more  efficient 
and  productive 
o  Cost  effectiveness 
o  Reduced  funds,  fuel,  and  ammunition 


for  military  operations  and  readiness 
exercises 

The  OoO  community  has  initiated  three  service 
efforts  which  focus  on  front-end  analysis.  The 
efforts  Include  the  Air  Force  Logistics  Composite 
Model  (used  primarily  for  maintenance  manpower 
modeling),  the  Army's  Personnel  Affordability 
program,  and,  of  particular  interest,  the  Navy's 
HARDMAN  program.  HARDMAN  (Military  Manpower 
versus  Hardware  Procurement)  is  being  designed 
to  identify  manpower,  personnel,  and  training 
constraints  as  early  as  Milestone  0. 

To  date,  there  has  been  little  motivation  for 
program  managers  to  take  into  consideration  the 
life  cycle  impact  of  manpower  on  the  system. 

The  manpower,  training,  and  training  equipment 
considerations  were  coming  too  late  in  the 
acquisition  cycle  to  influence  hardware  and 
software  planning.  HARDMAN  takes  personnel 
and  training  into  consideration  at  Milestone  0 
as  alternatives  during  the  Milestone  I  trade¬ 
off  analysis. 

As  I  mentioned  earlier,  an  actual  pvjgram  used 
as  an  example  would  be  of  some  help  in  pulling 
some  of  these  ideas  together.  The  Navy's 
VTXTS  program  is  an  excellent  example  of  a 
major  system  that  is  in  the  acquisition  cycle 
and  has  an  approved  MENS.  The  VTXTS  program 
also  used  HARDMAN  to  analyze  the  personnel 
requirements  for  the  VTX  aircraft.  Becuase  of 
this,  manpower  data  was  available  for  Mile¬ 
stone  I  trade-off  evaluation  -  valid  trade-offs 
could  be  made. 

The  Undergraduate  Jet  Flight  Training  System 
(VTXTS)  is  a  multi-faceted  training  system 
which  will  be  used  in  the  intermediate  and 
advanced  phases  of  the  Naval  Flight  Training 
Program  to  train  pilots  in  the  operation  of 
jet  aircraft.  The  VTXTS  is  to  be  developed  as 
an  integrated  system  of  four  mutually  support¬ 
ive  major  elements.  These  four  elements  are: 
academics,  simulators,  aircraft  and  training 
management  system. 

Let's  now  go  thru  the  VTXTS  MENS  to  gain  a 
better  understanding  of  the  elements  of  a  MENS 
and  how  they  relate  to  personnel  (see  Table  2  - 
VTXTS  MENS  OUTLINE). 

TABLE  2  VTXTS  MENS  OUTLINE 

1.  Mission 

2.  Basis  of  Need 

3.  Existing  and  Planned  Capabilities 
to  Accomplish  this  Mission 

4.  Assessment 

5.  Constraints 

6.  Program  Plan  to  Identify,  Explore 
and  Evaluate  Alternative  Training 
Concepts 
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Some  highlights  from  the  MENS  are:  Under  Mission 
"The  overall  mission  area  encompasses  the  contin¬ 
uing  requirement  to  provide  undergraduate  pilot 
training.  .  .  A  need  exists  to  extend  or  provide 
an  optimized  replacement  for  the  present  training 
system  to  meet  future  pilot  production  require¬ 
ments."  For  the  VTXTS  program  it  can  be  seen 
that  the  mission  itself  is  people  oriented. 

The  next  element  is  the  Basis  of  Need.  Again  we 
can  see  a  people  influence.  "The  training  of 
sufficient  numbers  of  jet  pilots  is  vital." 

The  third  element  is  that  of  Existing  and  Planned 
Capability  to  Accomplish  the  Mission  and  it  is 
broken  down  as  follows: 

A.  Existing  System  Capability  -  here  a 
short  description  of  the  current 
Navy  undergraduate  pilot  training 
system  was  given. 

B.  DoD  and  Allied  Capability  -  in  this 
area  it  was  stated  that  other  sources 
do  not  satisfy  the  requirements. 

C.  Agencies  and  Components  Involved  - 
The  U.S.  Navy  and  Marine  Corps  and 
possibly  NATO  and  the  U.S.  Air  Force 

In  the  area  of  Assessment  the  VTXTS  had  several 
i terns : 

The  first  item  consisted  of  a  short  descrip¬ 
tion  of  the  Training  System  Concept  Require¬ 
ments. 

The  second  item  identified  deficiencies 
in  the  existing  capability  and  of  partic¬ 
ular  interest  the  statement  "The  existing 
Navy  flight  training  system  is  becoming 
increasingly  expensive  to  operate  and 
maintain.  Escalating  personnel,  material 
and  fuel  costs  are  driving  ever  higher 
student  costs-to-train. "  It  was  also 
mentioned  that  the  hardware  was  becoming 
obsolete  but  this  did  not  receive  as  much 
attention  as  the  personnel  problem. 

The  last  item  to  be  assessed  was  the 
Opportunity  for  Cost  Savings.  Here  it  was 
discussed  that  a  wide  range  of  options 
would  be  evaluated  to  seek  an  integrated 
system  taking  full  advantage  of  state-of- 
the-art  simulation,  and  trainers  including 
the  Flight  Vehicle. 

The  next  major  area  to  be  covered  was  the  Con¬ 
straints  to  the  System.  The  constraints  in¬ 
cluded  carrier  operability,  cost,  timing  for 
IOC,  NATO  Standardization  and  Interoperability 
and  finally  the  constraints  of  manpower,  per¬ 
sonnel  and  training.  "Design  proposals  for  the 
system  will  include  an  assessment  of  life  cycle 
costs  and  those  potential  life  cycle  cost  trade¬ 
offs  between  hardware  capability  and  available 
manpower  and  logistic  support.  The  basic  man¬ 
ning  concept  for  the  flight  training  system  is 
Intended  to  Influence  system  design  to  achieve 
the  most  cost  and  training  effective  military, 
civilian  and  contractor  manpower  mix." 

The  next  major  area  addressed  the  "Impact  of 
Staying  with  the  Present  System."  The  basic 


statement  here  was  that  the  mission  needs  cannot 
be  met  with  the  existing  system. 

The  last  area  was  the  Program  Plan  for  Alter¬ 
native  Training  Concepts.  This  covered  the 
overall  plan,  the  analysis  effort,  the  assign¬ 
ment  of  a  program  manager  and  finally  projected 
Milestones. 

As  you  can  see,  VTXTS  is  an  example  of  best  case 
or  worst  case,  depending  on  how  you  look  at  it, 
of  a  people  oriented  MENS.  Since  personnel  and 
training  were  a  part  of  the  mission,  the  need, 
the  deficiency,  and  the  solution  this  was  not 
exactly  your  typical  MENS.  It  does  however 
provide  some  food  for  thought  on  how  people 
should  be  a  part  of  any  MENS. 

Let's  examine  another  key  factor  in  determining 
why  personnel  must  be  taken  into  consideration 
in  mission  needs.  The  personnel  available 
today  in  the  military  work  force  is  significant¬ 
ly  different  in  quantity  and  ability  than  it 
was  when  most  of  our  current  systems  started 
into  the  acquisition  process.  What  constituted 
the  weapon  system  capable  of  fulfilling  a 
mission  then  might  no  longer  exist.  The  design 
of  any  system  ultimately  must  include  people  -  - 
either  as  operators  or  as  maintainers.  If  the 
people  available  now  as  part  of  the  system  are 
not  the  same  as  those  projected  as  being  avail¬ 
able  during  system  design,  there  is  a  probability 
the  system  will  not  meet  the  total  mission 
element  needs.  If  the  prerequisite  tools  ex¬ 
pected  of  the  human,  e.g.  reading  level,  motiva¬ 
tion,  electronic/mechanical  aptitude,  are  not 
available  in  the  quanity  of  people  required, 
then  a  deficiency  exists.  In  this  case,  the 
mission  element  need  is  expressed  and  assessed 
as  a  deficiency  in  the  force  available  for  the 
system.  Trainers,  part  task  trainers,  and 
generic  ski  11  s/competency  trainers  can  help 
fulfill  the  need. 

The  preceding  example  depicted  how  people  can 
be  expressed  in  a  MENS  as  an  assessment  of  a 
deficiency.  People  must  always  constitute  a 
constraint  —  if  it  is  accepted  that  the  Per¬ 
sonnel  Subsystem  constitutes  a  major  component 
of  a  system.  The  availability  and  quality  of 
a  major  subsystem  must  constitute  a  constraint. 
Again,  it  shoud  be  emphasized  that  in  the  past, 
the  human  was  only  a  small  part  of  the  total 
system.  A  current  system  is  considered  to  be 
composed  of  a  Hardware  Subsystem,  a  Software 
Subsystem,  and  a  Human  or  Personnel  Subsystem, 
see  Figure  2  -  A  TOTAL  SYSTEM.  Any  system 
may  be  composed  of  any  or  all  subsystems,  or 
a  mixture  of  subsystems  in  any  ratio.  Looking 
at  the  pie-diagram  on  the  left,  we  can  remember 
when  people-costs  (labor-costs)  were  given 
only  incidental  consideration  in  comparison 
with  the  Investment  of  new  hardware.  A  couple 
of  things  have  occurred  by  this  point  in  time: 

1.)  Labor-costs  have  grown  at  a  phenomenal 
rate  relative  to  the  cost  of  machines.  This 
however  Is  just  the  tip  of  the  Iceberg  for  the 
military.  If  one  compares  military  pay  to 
civilian  pay  for  example: 
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o  Civilian  starting  salaries  are  SOS  over 
2nd  IT's 

o  Enlisted  average  salary  is  $9,900 

o  Staff  Sgt  with  10  years  =  Washington 
O.C.  Unemployment 
o  30%  Below  Minimum  Wage 
We  must  assume  that  the  military  pay  rate  will 
increase  at  an  even  greater  rate,  and  that  this 
increase  will  cause  the  people-costs  for  a 
system  to  be  even  more  significant. 


HUMAN 


SVSTEM  ENVIRONMENT  SYSTEM  ENVIRONMENT 

l FROM  THE  PAST |  (TODAY! 


Figure  2  A  TOTAL  SYSTEM 


2.)  Computer  power  has  vastly  extended  the  lev¬ 
erage  of  man  over  machine  and  his  environment. 
The  pie  diagram  on  the  right  better  describes 
the  system  considerations  of  today.  The  system 
conceptualist,  in  synthesizing  the  alternative 
ways  to  meet  the  operational  requirements,  has 
potentially  infinite  options  in  allocating 
functions  among  the  three  subsystems. 

People  -  -  not  dollars  -  -  are  going  to  be  the 
ultimate  limiting  resource  of  future  systems. 
This  is  obvious  from  at  least  two  factors: 

1.)  the  cost  of  people  constitutes  over  half  of 
the  DoO  budget,  and  the  percentage  is  growing 
each  year.  2.)  Seventy  percent  of  the  decisions 
that  affect  life  cycle  cost  are  made  by  OSARC  I. 

CONCLUSION 

In  conclusion,  one  must  consider  the  cost, 
quality,  and  availability  of  the  projected  labor 
pool  as  part  of  the  MENS  preparation  effort. 

The  life  cycle  cost  and  viability  of  the  threat- 
responsive  system  can  be  significantly  impacted. 
This  will  force  the  requirement  for  trade-offs 
between  1.)  the  man,  hardware .and  software  sub¬ 
systems  and  2.)  and  part- task  or  part-mission 
trainers  and  large  full  mission  simulators  which 
minimize  the  cost  of  procuring  and  supporting 
the  total  trainer  package.  It  will  also  ensure 
greater  concurrency  between  the  configuration 
of  the  weapon  system  and  that  of  the  trainer 
suite. 
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ABSTRACT 


This  article  deals  with  the  “buying-in"  syndrome.  Buying-In  describes  the  contractor's 
actions  In  trying  to  win  a  contract  award.  In  short,  the  government  sets  a  trap  for  itself. 
Then  the  contractor  bids  below  the  actual  costs  in  anticipation  of  being  awarded  a  sole-source 
contract  where  losses  are  recovered.  Government  funding  characteristics  even  encourage  this 
action. 
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INTRODUCTION 


From  one  perspective,  three  distinct  elements 
integrate  to  produce  what  we  call  the  buying-in 
syndrome.  First  the  government  sets  a  trap  for 
itself.  Second  the  contract  costs  reported  to 
the  government  soar,  especially  on  follow-on 
contracts.  Third,  government  funding  cycles  and 
obligation  policies  encourage  contractor  activit¬ 
ies  that  aggravate  the  situation.  This  paper 
deals  with  the  segments  separately,  with  the  first 
segment  establishing  the  framework  for  integrat¬ 
ing  the  other  two.  "Buying-in"  is  a  term  that 
has  been  bandied  about  in  the  acquisition  business 
for  as  long  as  I  can  remember.  Some  government 
officials  see  buying-in  as  a  sinister  plot  by 
contractors  to  secure  a  government  contract  under 
false  pretenses.  Some  contractors  see  buying-in 
as  bidding  in  the  fashion  that  has  become  stand¬ 
ard  in  the  face  of  unachieveable  government 
technical  and  schedule  requirements.  Another 
view  is  that  company  design,  development  and 
production  techniques  are  so  efficient  that  If 
enough  units  are  ordered,  the  contractor  will  pay 
the  government  to  take  the  last  few  units. 

Simulator  SPO  data  indicates  a  sort  of 
pattern  of  activities  that  comprise  a  situation 
that  becomes  tagged  as  a  buy-in.  The  situation 
consists  of  contractor  bids  that  are  nearly  equ  . 
to  the  government  budget,  with  schedules  equal 
to  the  government  schedule.  Best  and  Final 
Offers  when  received  are  significantly  lower  than 


the  initial  bid.  Very  shortly  after  contract 
award  an  over  target  condition  develops,  followed 
by  anticipated  schedule  slips.  The  end  result 
for  the  SPO  Director  an  overrun  contract,  late 
delivery  and  a  short  jf  private  audiences 

with  the  Cocrmander.  situation  doesn't  start 

with  the  contractors'  b.as.  The  bids  simply 
serve  to  spring  the  trap  that  the  government  has 
already  set  for  itself. 

Our  trap  is  constructed  by  digging  a  hole 
for  ourselves,  making  a  camouflage  cover  for  it, 
and  then  lining  it  with  sharp  stakes  so  that  when 
we  fall  in  we  get  hurt.  Our  major  excavation 
tool  is  the  proposed  delivery  date.  Once  this 
usually  unrealistic  date  is  Ironclad,  we  use  it 
to  force  short  suspenses  which  result  in  in¬ 
adequate  planning,  resultant  lower  quality  RFPs 
and  Insufficient  proposal  analysis.  The  net 
result  is  that  the  government  starts  in  a  hole. 

The  next  step,  preparing  a  camouflage  cover, 
follows  almost  Inexorably  from  the  first.  Once 
we  have  dug  the  hole  of  inadequate  preparation 
with  the  ironclad  delivery  date,  we  proceed  to  a 
frenzy  of  activity  which  appears  designed  to 
prove  that  there  really  is  no  hole.  The  force 
field  that  surrounds  this  activity  is  tremendous 
and  grossly  unbalanced.  Institutional,  career, 
and  bureaucratic  forces  combine  to  develop 
momentum  which  overwhelms  personal  forces  of 
principle,  integrity  and  craftsmanship  as 
illustrated  in  figure  I. 


This  mismatch  generates  the  Illusion  that  there  Is 
no  hole.  How  could  there  be:  All  suspenses  were 
met.  All  presentations  were  given.  Costs  and 
schedules  have  been  developed....  At  this  point 
the  camouflage  blanket  (about  1  micron  thick)  has 
been  developed  to  throw  over  the  hole  as  soon  as 
we  put  In  the  sharp  stakes.  The  sharp  stakes  are 
all  the  techniques  we  employ  to  make  absolutely 
sure  that  falling  in  the  hole  will  hurt  us.  Three 
of  the  most  predominant  are  (1)  basing  other  major 
programs  on  the  success  of  the  one  we  have  just 
established  under  the  1  micron  thick  camouflage 
blanket,  (2)  only  contracting  for  the  first  few 
units,  leaving  the  remainder  to  be  purchased  on  a 
follow-on  sole  source  contract,  and  (3)  providing 
very  small  positive  Incentives  for  outstanding 
contractor  performance. 

Now  that  we  have  successfully  laid  the  trap, 
the  camouflage  activity  begins  to  gain  strength 
from  the  success  of  Its  past  accomplishments  and 
effectively  blindfolds  everyone  who  comes  along. 
For  example,  if  there  Is  no  hole  there  is  no 
reason  to  discuss  It  with  appropriate  management. 
Situations  with  the  potential  for  becoming  serious 
problems  are  either  Ignored  or  put  under  the  head¬ 
ing  of  contractor  screw-ups  to  be  dealt  with  by 
"holding  his  feet  to  the  fire".  The  program  be¬ 
comes  the  child  of  the  SPO  and  therefore  not  only 
can  do  no  wrong  but  should,  at  the  drop  of  a  hat, 
be  defended  to  the  death.  Obviously  under  these 
circumstances  only  the  most  optimistic  forecasts  . 
are  to  be  believed  and  any  indications  to  the 
contrary  should  be  shelved  until  all  the  facts  are 
absolutely  established.  The  bl indfold  becomes 
more  and  more  effective  as  questions  arise  con¬ 
cerning  the  program.  More  questions  generate  more 
briefings  which  generate  more  briefing  charts, 
the  preparation  of  which  uses  the  time  which  would 
otherwise  have  been  spent  on  program  analysis. 

This  analysis  is  dangerous  because  it  could  pene¬ 
trate  the  1  micron  camouflage  blanket  and  expose 
the  hole  full  of  sharp  stakes. 

In  this  syndrome,  analysis  Is  not  desirable 
because  It  could  cast  doubt  on  the  viability  of 
the  program.  It  is  certainly  unnecessary  since 
a  1%  award  fee  is  hanging  over  the  contractor's 
head  to  make  him  do  everything  we  wanted.  And  if 
there  are  any  problems  the  contractor  can  just  go 
to  ceiling  and  get  paid  for  cleaning  up  the  de¬ 
tails.  Typical  management  steps  which  surma rlze 
the  orientation  that  exists  throughout  this 
sequence  of  activities  are: 

1.  Maintaining  that  there  is  no  development 
in  the  program,  just  bolting  together  some  off 
the  shelf  hardware  and  coming  up  with  some  soft¬ 
ware. 

2.  Oon't  surface  it  yet!  It  might  kill  the 
program. 

3.  Review  3  month  old  data  to  see  how  the 
author  of  the  data  says  we  got  where  he  says  we 
are. 

4.  Use  this  data  to  prepare  the  charts  and 
backups  and  have  two  prereviews  and  six  dry  runs 
to  check  format,  typing  and  graphics  quality. 

5.  Prepare  300  charts  each  quarter  for  the 
review  and  approval  cycle. 

6.  Be  sure  that  at  least  90%  of  the  review 
time  is  spent  on  the  1%  of  the  program  that 


everyone  feels  comfortable  discussing. 

The  buying-in  syndrome  is  further  aggravated 
in  the  government  by  the  review  and  approval  pro¬ 
cess.  Actual  comments  taken  out  of  their  original 
context  are  grouped  here  to  quickly  create  the 
buying-in  syndrome  perspective  of  the  review  and 
approval  process. 

“Your  charts  are  not  graphics  quality." 

Practically  the  only  comment  made  before  can¬ 
cellation  of  a  complex  briefing  on  a  series  of 
integrated  problems  with  a  potential  impact  of 
nearly  $1  billion. 

"I  don't  care  about  foreign  disclosure! 

How  could  you  be  so  far  behind?" 

A  comment  made  by  one  of  the  people  responsible 
for  a  one  year  delay  in  the  program  due  to 
inability  to  resolve  the  foreign  disclosure 
question . 

"Yes,  you  briefed  that  at  each  review, 

but  that  is  not  the  official  channel. 

Concurrence  at  the  review  is  not 

approval !" 

Part  of  the  refusal  to  accept  "review  and 
approval"  as  justification  for  a  course  of 
action  that  proved  to  be  inappropriate. 

"The  formats  are  changed  for 

tomorrow's  briefing." 

"I  think  what  the  General  wants..." 

"Those  were  the  wrong  formats!" 

Dally  business. 

*  -  The  buying-in  syndrome  as  it  exists  in  the 
government  side  of  the  house  is  not  a  conscious 
malicious  overt  act  of  commission  or  omission. 

It  is  an  almost  infinite  series  of  human  actions 
conditioned  by  years  of  stimuli  for  which  certain 
responses  like  "can  do!"  are  expected  or  even 
demanded . 

This  demand  is  applied  by  the  almost  irresis- 
table  Inertial  force  of  the  bureaucratic  system 
pushing  toward  mediocraty.  Stated  simply  it  is 
the  infinite  compounding  of  each  individual 
assertion  that  there  really  is  a  free  lunch. 

The  next  two  elements  illustrate  the  cir¬ 
cumstances  that  are  viewed  as  a  buy-in  and  how 
contractor  financial  practices  and  government 
funding  policies  can  be  integrated  to  nurture 
the  syndrome. 
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This  following  illustration  (see  figure  II) 
is  an  actual  example  of  an  apparent  “buy-in" 
using  figures  from  a  recent  contract. 

The  situation  is  a  basic  contract  for  two 
units  with  an  option  for  four  more,  followed 
by  a  sole-source  contract  for  six  additional 
units. 

The  learning  curve  expressed  in  the  contrac¬ 
tual  values  of  the  first  six  units  is  30%  steeper 
than  any  learning  curve  ever  experienced  in  the 
history  of  the  simulator  program  office.  This 
unheard  of  learning  curve  is  characteristic  of 
a  "buy-in".  The  competition  surrounding  the 
basic  contract  forces  the  contractor  to  make 
unrealistic  bids  in  order  to  win  the  contract 
award.  The  contractor  is  therefore  dependent 
on  the  sole-source  (non-competitive)  contract 
to  recover  losses  suffered  on  the  basic. 

If  you  are  wondering  how  a  contractor  can 
survive  and  even  prosper  while  actual  costs  are 


over  contractual  celling  on  the  basic  contract, 
the  following  will  explain  how  government  funding 
characteristics  make  it  possible. 

As  the  contractor  begins  work  on  the  first 
two  units,  billings  are  submitted.  Being  that 
the  contractor's  costs  are  higher  than  contractual 
amounts,  billings  would  soon  exceed  the  funding 
level  causing  expenses  to  be  out  of  the  contrac¬ 
tor's  pocket.  For  the  contractor,  the  life¬ 
saving  aspect  is  that  the  option  is  exercised 
by  the  government  before  the  first  two  units  are 
completed  and  before  the  contractor's  costs  have 
exceeded  the  funds  available.  With  the  exercise 
of  the  option  comes  an  increase  in  the  funds 
available.  This  incident  happens  again  several 
months  later,  before  the  option  units  are  com¬ 
pleted,  when  the  government  awards  a  sole- 
source  contract  for  six  additional  units  and 
again  raising  the  funds  available. 


OPTION 


BASIC 


SOLE  SOURCE 


II.  The  Buy-In 
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SIMULATOR  QUALIFICATION  TESTING-SHARING  THE  RISKS 
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ABSTRACT 

The  qualification  testing  of  most  Air  Force  aircrew  simulators  has  stretched  out  far  beyond 
the  original  test  schedule  estimates.  The  result  has  been  additional  costs  and  extensive  delays  in 
getting  the  simulators  into  the  field  in  a  ready-for-training  status.  This  paper  examines  the 
historical  data,  identifies  the  persistent  problem  areas,  assesses  the  inherent  risks,  and  offers 
some  recommendations  for  reducing  the  risks  to  both  the  government  and  the  contractor. 


INTRODUCTION 

The  qualification  testing  period  of  Air  Force  simu¬ 
lators  has  historically  been  a  very  troublesome  period  in 
the  acquisition  cycle  for  both  the  contractor  and  the  Air 
F orce.  Qualification  testing  is  the  Air  Force  conducted 
test  and  evaluation  of  simulators  which  is  the  equivalent 
of  development  test  and  evaluation  (DT&E)  in  other 
acquisition  programs.  It  is  the  "proof  of  the  pudding" 
where  we  rigorously  determine  the  contractor's  com¬ 
pliance  with  the  product  specification. 

When  the  time  comes  to  start  qualification  testing, 
the  contractor  is  aware  that  problems  still  exist,  but  he 
plans  to  keep  a  few  steps  ahead  of  the  Air  Force  team 
rather  than  delay  the  start  of  testing.  As  testing  pro¬ 
gresses,  the  contractor  comes  under  increasing  pressure 
to  stay  ahead  of  the  Air  Force  test  team.  Not  only  must 
he  fix  the  known  discrepancies  before  the  team  gets  to 
that  area  of  test,  but  he  must  also  fix  the  newly  dis¬ 
covered  problems  and  maintenance  difficulties  which  fall 
into  the  category  of  immediate  "showstoppers," 
Experience  shows  that  the  contractor  team  is  sooner  or 
later  overcome  by  events,  and  testing  grinds  to  o  slow 
walk  and  may  even  be  forced  to  halt.  This  can  hardly  be 
o  more  inopportune  time  since  the  schedule  reserves  are 
probably  all  gone  and  the  contractor  is  struggling  just  to 
preserve  some  portion  of  the  profit  he  had  originally 
counted  on.  At  this  point  the  "finger  pointing"  starts  or 
increases  in  intensity  and  the  whole  mess  is  exacerbated 
by  strong  adversary  relationships  between  the  govern¬ 
ment  and  contractor.  These  in  turn  are  further  aggra¬ 
vated  by  frustration,  16  hour  work  days,  and  red  ink  on 
the  bottom  line.  We  eventually  pick  up  the  pieces  and 
get  back  on  track,  but  not  without  paying  the  price  of  a 
nonrecoverable  schedule  slip. 

Obviously,  this  all  too  familiar  scenario  places  both 
the  Air  Force  and  the  contractor  in  undesirable  posi¬ 
tions.  The  contractor  is  motivated  to  minimize  his 
losses  while  still  getting  the  system  out  the  door.  It  is 
easy  for  an  error  in  judgment  to  result  in  prolonging  the 
agony  through  the  reduction  of  project  resources  and  use 
of  short  cuts  at  a  time  when  just  the  opposite  approach 
should  be  taken.  The  Air  Force,  on  the  other  hand,  must 
tell  users  and  the  higher  headquarters  mtragers  that  the 
much  needed  training  system  will  not  be  available  as 
scheduled.  This  notification  must  be  explained  and 
justified  up  to  the  Air  Force  Council  because  of  the 
impact  an  Air  Force  operational  costs  and  readiness.  It 
is  easy  to  understand  why  the  delay  at  the  point  of 
qualification  tests  results  in  letters  of  direction,  claims, 
and  other  legal  actions— all  of  which  both  the  Air  Force 
and  the  contractors  would  like  to  avoid. 


THE  PROBLEM 

In  order  to  better  define  the  problem,  some  research 
into  the  historical  records  of  the  Air  Force  Simulator 
Systems  Program  Office  (SPO)  has  been  accomplished. 
Data  were  surveyed  on  some  18  programs  which  have 
been  or  are  being  managed  by  the  Simulator  SPO.  In 
each  case  we  compared  the  planned  qualification  test 
period  to  the  actual  time  spent,  or  in  the  case  of  the 
ongoing  programs,  the  time  spent  to  date. 

The  results  for  the  completed  or  mature  programs 
are  shown  in  Figure  I.  Note  that  in  all  cases  except 
three,  the  qualification  test  periods  exceed  the  planned 
test  periods,  in  most  cases  by  a  significant  amount. 
Although  this  result  was  not  surprising,  it  was  rather 
sobering  to  view  our  very  poor  track  records  on  a 
collective  basis. 
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Those  of  you  who  have  dealt  with  statistics  and  the 
necessary  analysis  to  support  meaningful  conclusions 
know  that  the  next  step  was  to  examine  each  program 
and  ask  the  question,  "Why  did  qualification  testing  ex¬ 
tend  beyond  the  planned  period?"  In  an  attempt  to 
answer  this  question,  several  factors  were  considered: 

-Number  of  SRs/TDs  (Service  Reports/ Test  Discrep¬ 
ancies)  written 

-Test  versus  fix  time 
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-Requirements  definition 
-Data  base  inadequacies 
-Quality  of  hardware  design 
-Quality  of  software  design 
-Sptre  core/ time  deficiencies 
-Test  readiness 

-Contractor  support  during  test 
-Government  management/support 
-Subjective  tweaking 

Of  the  above  considerations,  the  first  two,  number  of 
SRs/TDs  and  test  versus  fix  time,  lend  themsleves  to 
quantification.  The  others  are  judgment  colls  end  sub¬ 
ject  to  disagreement,  depending  on  perspective.  There¬ 
fore,  our  dialysis  treated  the  factors  in  just  that  fashion. 
On  the  judgment  side,  we  reviewed  the  records  and 
talked  to  the  Air  Force  program  managers  to  reach  a 
conclusion.  It  should  also  be  noted  that  there  is  no 
intention  to  publish  judgmental  data  which  is  critical  or 
reflects  poorly  on  the  performance  of  any  contractor. 
Consequently,  our  conclusions  on  the  judgmental  factors 
are  not  aligned  by  program. 

Figure  2  is  a  plot  of  the  number  of  SRs/TDs  versus 
the  qualification  test  period.  The  diagonal  lines  are 
least  square  lines  approximating  the  PTT/CPT  (Part  Task 
Trainer/ Cockpit  Procedures  Trainer)  devices  and  the 
WST/OFT/FS  (Weapon  Systems  Trainer/Operational 
Flight  Trainer/lnstrumenf  Flight  Simulator)  devices.  As 
would  be  expected,  the  more  complicated  simulators 
were  subject  to  more  writeups  than  the  less  complex 
PTTs  and  CPTs.  It  would  appear  in  general  that  the 
systems  which  require  more  fixes  must  stay  in  test 
longer.  Of  course  one  can  also  argue  that  the  longer  a 
device  is  in  test,  the  more  writeups  it  receives.  The 
noticeable  difference  between  the  nominal  CPT/PTT  line 
and  the  OFT/IFS/WST  line  is  due  to  the  relative  com¬ 
plexify  of  the  systems,  and  may  also  be  driven  in  some 
cases  by  deficiencies  in  the  design  data  base.  It  is 
worthwhile  to  note  that  the  SEWT  (Simulator  for 
Electronic  Warfare  Training)  and  KC-135  80PTT  (Boom 
Operator  Part  Task  Trainer),  two  of  our  on-time 
systems,  had  relatively  few  writeups.  Even  so,  we  did 
not  find  these  dota  in  themselves  to  be  conclusive,  nor 
did  they  reveal  any  results  that  drive  us  toward  specific 
corrective  recommendations, 

TEST  (MMMIKKS-VS-1MC 
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The  comparison  of  test  time  to  fix  time  turned  out  to 
be  somewhat  more  interesting  and  more  conclusive. 
Figure  3  reveals  that  the  octual  time  spent  in  test 
closely  approximates  the  planned  qualification  test 
period  in  3  out  of  S  cases.  Since  we  have  not  previously 
recorded  the  individual  test  and  fix  times  specifically, 
we  used  only  those  programs  for  which  the  data  could  be 
gleaned  from  the  records.  It  is  believed  that  the  close 
approximation  between  actual  test  time  and  planned  test 
time  supports  our  original  estimate  for  total  qualific¬ 
ation  test  period  where  the  assumption  has  always  been 
mode  that  the  system  will  have  successfully  completed 
contractor  testing  and  DCAS  observed  testing  prior  to 
going  into  Air  Force  qualification  testing,  and,  there¬ 
fore,  will  be  largely  fault  free.  As  we  examine  the 
records  and  our  own  experience,  however,  we  must 
conclude  that  our  basic  premise  is  false.  Contractors  do 
not  always  perform  thorough  independent  QTP  systems 
tests  and  the  DCAS  role  is  more  QC  oriented  than  per¬ 
formance  oriented. 
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Figure  4  serves  as  a  synopsis  of  the  judgment  factors 
as  they  were  evaluated  in  six  programs.  The  question  we 
asked  ourselves  in  each  case  is  whether  the  factor  had  an 
impact  on  the  extension  of  the  qualification  testing 
period.  In  all  programs,  the  lack  of  test  readiness  and 
problems  in  software  design  were  schedule  drivers.  You 
will  note  that  lack  of  contractor  support,  which  was 
defined  as  the  lack  of  qualified  personnel  at  the  right 
time  or  in  the  correct  numbers,  also  ranked  high  as  a 
schedule  extender.  It  is  believed,  however,  that  lock  of 
test  readiness  made  the  personnel  problem  more  acute 
than  would  normally  be  the  case.  Surprisingly,  the  sub¬ 
jective  tweaking  problems  were  seen  as  significant 
drivers  in  only  three  out  of  the  six  programs  surveyed.  It 
must  be  noted,  however,  that  when  subjective  testing 
was  relied  on  as  a  major  part  of  making  the  simulator 
perform  in  an  acceptable  fashion,  it  had  a  substantial 
impact  on  the  schedule.  The  collection  and  correct  use 
of  the  data  base  was  considered  a  significant  factor  in 
four  out  of  the  six  cases.  All  of  these  four  programs 
involved  aerodynamic  qualities  end  three  of  these  four 
also  suffered  in  test  schedule  because  of  subjective 
tweaking. 

In  summary,  it  is  suggested  that  test  reodiness  and 
associated  contractor  support  rank  as  major  problems. 
The  data  base,  software  design,  and  subjective  tweaking 
appear  to  be  interdependent  problems  with  major 
impact.  We  will  consider  these  problems  in  our  discuss¬ 
ion  of  risk  and  in  our  recommendations  for  improvement. 

RISK  ASSESSMENT 

The  uncertainty  of  the  time  required  to  complete 
qualification  testing  presents  risks  to  both  the  contrac¬ 
tor  and  the  government.  These  risks  are  not  only  asso¬ 
ciated  with  the  test  period,  but  are  also  linked  back 
through  the  project  to  the  very  beginning.  Actions  of 
both  the  contractor  and  the  Air  Force  throughout  the 
life  of  a  project  are  in  many  ways  the  result  of  risk 
assessments  and  perceptions.  The  following  paragraphs 
present  some  analyses  end  insights  into  the  problem  of 
qualification  testing  as  viewed  from  the  perspective  of 
risks. 

Cost  is  obviously  an  area  of  risk  that  affects  both  the 
contractor  and  the  government.  The  costs  for  the  con¬ 
tractor  are  a  direct  function  of  the  time  in  test.  On  the 
other  side,  any  delays  run  up  the  government  costs  for 
test  support,  and  more  importantly,  delay  the  availab¬ 
ility  of  the  system  for  training.  From  the  contractor 
poirrt-of-view,  the  government  may  test  as  long  as 
desired,  provided  the  contractor  is  being  paid  for  the 
testing.  However,  since  most  contracts  are  FFP  or 
FPIF,  testing  beyond  the  planned  period  results  in  a  loss, 
or  at  best  a  shared  arrangement  due  to  the  incentive 
provisions  of  the  controct  (assuming  costs  fall  between 
target  and  ceiling  on  the  FPIF  contract). 

There  is  a  tendency  to  attack  the  overall  problem  of 
risk  from  a  very  limited  perspective.  Industry  would  feel 
comfortable  with  a  cost  reimbursement  arrangement  for 
the  test  period.  However,  that  would  place  all  the  risk 
on  the  government.  On  the  other  hand,  a  FFP  contract 
shifts  all  risk  to  the  contractor.  From  the  contracting 
view,  the  risk  should  be  proportional  to  the  ability  of 
each  p<rty  to  control  that  risk. 

Test  cannot  be  viewed  to  be  bounded  by  just  that 
time  period  during  which  the  qualification  test  occurs. 
It  is,  indeed,  much  brooder  than  that.  The  success  or 
failure  during  qualification  test  starts  at  the  very  be¬ 
ginning  of  the  formulation  of  requirements  end  does  not 


end  until  the  completion  of  the  qualification  test  period. 
What  we  must  do  in  considering  the  risk  is  to  take  a 
systems  approach,  e.g.,  look  at  each  element  of  the 
acquisition  process  that  affects  test  and  assess  the 
individual  and  collective  impocts  on  risk. 

Requirements  Definition 

The  definition  of  training  requirements,  hopefully 
through  the  ISD  (Instructional  Systems  Development) 
procedure,  becomes  the  basis  for  the  determination  of 
the  system  to  be  built.  The  training  requirements  drive 
us  toward  a  specification  which  describes  a  device  some¬ 
where  along  the  continuum  from  a  PTT  to  a  full  WST. 
The  intended  use  of  the  device  in  training  may  necessi¬ 
tate  a  full  fidelity  approach  or  may  allow  little  resembl¬ 
ance  between  the  trainer  and  the  actual  operational 
hardware.  It  is  the  full  fidelity  trainer  that  this  paper 
will  emphasize  since  the  cxithors  have  little  experience 
with  the  other  types  of  trainers.  Furthermore,  it  is 
trainer  fidelity  that  we  normally  strive  for,  and  it  is  this 
same  fidelity  that  is  the  source  of  the  risk  for  both  the 
government  and  the  contractor.  A  large  portion  of  our 
testing  activity  is  to  prove  that  the  simulator  looks, 
feels,  and  performs  just  like  the  operational  hardware. 
One  of  our  favorite  specification  references  reads, 
"Simulator  performance  characteristics  shall  not  be  per¬ 
ceptibly  different  from  characteristics  of  the  real  world 
aircraft." 

If  the  performance  of  the  operational  hardware, 
which  we  seek  to  simulate,  was  fully  defined  in  quantit¬ 
ative  engineering  terms  with  no  areas  of  uncertainty,  the 
risks  in  designing,  building,  and  testing  a  simulator  would 
be  simply  those  associated  with  the  application  of  approp¬ 
riate  existing  technology  to  achieve  the  desired  result. 
These  are  the  risks  which  every  contractor  must  take  in 
doing  business.  However,  the  risk  in  achieving  accept¬ 
able  performance  (within  planning  estimates)  increases 
dramatically  when  the  performance  of  the  device  is 
poorly  defined,  or  in  some  cases  not  defined,  and  the 
contractor  must  guess  at  what  the  final  performance 
should  be. 

Data  Bose 

The  data  base  for  simulator  design  is  usually  the 
responsiblity  of  the  contractor.  He  is  required  to  collect 
existing  data  from  any  available  source  in  order  to 
firmly  establish  the  design  basis  for  the  simulator.  We 
assist  in  this  collection  process  by  requiring  that  the  air¬ 
frame  contractors  supply  data  to  the  simulator  contr¬ 
actors.  However,  the  required  data  are  not  always 
complete  or  in  a  format  which  is  usable  by  the  simulator 
contractors.  These  deficiencies  result  in  risks  to  the 
simulator  contractors  since  the  quality  of  the  data  can 
only  be  ascertained  through  detailed  t»Hilysis— a  process 
which  seldom  occurs  during  the  proposal  stage. 

It  has  been  suggested  that  all  baseline  data  be 
gathered  and  certified  by  the  procuring  agency  to  relieve 
the  contractor  of  the  risk  over  which  he  has  only  limited 
control.  This  has  been  considered,  and  is,  in  fact,  being 
done  by  some  government  agencies.  However,  in  order 
for  the  government  to  collect  and  supply  the  data  to  the 
contractor,  a  team  of  engineers,  technicians,  and  clerks 
must  be  dedicated  to  the  task,  and  the  government  must 
assume  the  risk  associated  with  the  data  base  process. 
This  risk  in  turn  spills  over  into  the  design  process  sinct 
the  design  is  data  base  dependent.  It  is  our  opinion  that 
the  contractor,  as  the  simulator  desk; '  agency,  can 
perform  the  dato  task  in  a  more  efficient  manner,  and  in 
general,  should  be  held  responsible  for  obt-.ning  the  data 
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needed  for  design.  It  is  recognized,  however,  thot  should 
a  number  of  contractors  need  to  operate  from  a  common 
data  base  in  either  a  cooperative  venture  or  a  competi¬ 
tive  one,  it  may  be  appropriate  for  the  government  to 
directly  control  the  data  base. 

Design 

The  risk  associated  with  the  design  process  falls 
squarely  on  the  shoulders  of  the  contractor.  It  is  the 
contractors  expertise  in  this  area  that  serves  as  a  found¬ 
ation  for  his  business.  The  Air  Force  has  no  intention  of 
designing  simulators  for  which  industry  is  being  paid  to 
design.  This  statement  is  not  meant  to  deny  the  fact 
that  we  place  a  number  of  constraints  on  the  design.  But 
the  performance  of  the  simulator  during  test,  as  it 
relates  to  design,  is  a  contractor  responsibility  and  any 
performance  discrepancy  which  is  clearly  not  in  accord¬ 
ance  with  the  available  data  must  be  fixed  by  the  contr¬ 
actor. 

Qualification  Test  Procedure  (QTP) 

The  QTP  is  frequently  an  area  of  contention  between 
the  government  and  the  contractor,  and  is,  therefore,  a 
source  of  risk.  The  government  wishes  to  perform  a 
rigorous  series  of  tests  to  assure  that  the  simulator 
meets  every  requirement  of  the  specification.  Al  ihough 
the  contractor  agrees  in  principle,  there  is  a  tendeicy  to 
feel  that  a  less  thorough  test  program  will  accomplish 
the  same  objective.  The  preparation  of  the  QTP  is 
frequently  delayed  until  the  very  last  minute.  This  delay 
results  in  compression  of  the  effort,  and  inadequate  time 
for  review,  revision,  and  approval.  We,  therefore,  enter 
qualification  testing  without  full  definition  of  all  the 
tests  and  corresponding  success  criteria.  These  uncert¬ 
ainties  lead  to  disagreements  and  less  efficient  imple¬ 
mentation  of  testing. 

Test  Readiness 

The  test-fix-recheck  process  is  central  to  the  irob- 
lem  of  stretchout  of  fhe  qualification  test  period.  We 
presented  data  in  Figure  3  which  was  the  basis  for  our 
conclusion  that  our  test  estimates  are  not  all  that  bad, 
but  that  unscheduled  downtime  for  fixes  is  the  real 
culprit  in  the  schedule  slips.  In  other  words,  if  the 
machine  could  undergo  testing  without  the  resulting 
writeups,  our  test  estimates  would  be  fairly  accurate.  It 
is,  of  course,  unrealistic  to  expect  that  no  writeups  will 
be  necessary.  However,  if  a  system  has  been  thoroughly 
and  successfully  tested  by  the  contractor  and  DCAS 
prior  to  the  official  start  of  government  qualification 
testing,  the  testing  should  progress  smoothly  with  few 
interrupt  ions  to  correct  deficienciesor  equipment  failures. 

Tweaking 

Perhaps  the  biggest  risk  to  the  contractor  stems  from 
the  contract  requirements  to  make  it  fly  like  the  air¬ 
craft-par  ticularly  when  the  success  criteria  cannot  be 
established  on  a  firm  quantitative  basis.  What  we  must 
deal  with  then  are  sUbjecfive  evaluations  by  Air  Force 
test  personnel.  This  evaluation  process  is  obviously 
person  dependent  and  can  lead  to  numerous  tweaking  and 
redesign  efforts  because  of  non-repeatable  tests.  The 
separation  of  interactive  variables  becomes  cloudy  and 
very  dependent  on  the  capability  of  the  Air  Force  test 
pilot.  The  tests  can  easily  stretch  out  and  frustrations 
on  both  sides  run  high  since  progress  is  slow  and  there  is 
no  way  to  fully  establish  responsibli  ty  of  the  stretchout. 
Industry  has  accepted  this  risk  in  the  past,  but  cost  over¬ 


runs  have  reduced  the  willingness  to  continue  in  tnis 
fashion. 

RECOMMENDATIONS 

Having  examined  the  problem  and  the  risks,  it  is  now 
appropriate  that  we  consider  how  we  shoulu  approacn  tne 
test  period  in  the  future  to  better  assure  schedule 
compliance,  avoid  additional  unforeseen  costs,  ana  snare 
the  risks  in  a  more  equitable  fashion.  The  following 
paragraphs  address  and  briefly  explain  a  number  ot 
recommendations. 

Requirements 

It  seems  elementary  to  state  that  requirements 
should  be  defined  by  the  operational  commana  and  that 
these  requirements  should  be  clearly  developed  ana  re¬ 
flected  in  the  RFP.  However,  our  experience  has  shown 
that  much  work  still  needs  to  be  done  in  the  require¬ 
ments  area.  The  ISD  process  is  Deing  emphasized  ana 
further  defined  in  an  attempt  to  attack  the  proDlem. 
High  level  visibility  is  also  being  focused  through  the 
actions  of  the  Air  Force  Simulator  Advisory  Croup  ISAC; 
and  by  the  rejection  of  some  Program  Management 
Directives  (PMDs)  for  further  definition  before  action 
will  be  taken.  We  are  also  trying  to  improve  our  speci¬ 
fications  and  reduce  the  potential  for  misunderstandings. 
Our  program  teams  have  been  directed  to  avoid  informal 
changes  which  can  have  an  impact  on  cost  or  scneaule 
(including  test)  or  lead  to  later  disagreements,  claims, 
etc.  In  other  words,  we  must  use  existing  techniques  to 
formalize  all  changes.  Industry  needs  to  cooperate  fully 
with  us  in  this  attempt  to  avoid  test  problems  wnich  are 
requirement  related. 

Data  Bose 

The  proper  collection,  analysis,  and  utilization  ot  tne 
data  base  is  absolutely  essential  if  we  are  to  make 
significant  progress  in  reducing  the  risks  associated  with 
qualification  testing.  What  we  need  is  a  means  ot 
moving  from  qualitative  testing  toward  quantitative 
testing.  Through  proper  use  of  the  data  base  we  can 
quantify  the  expected  results  for  most  tests,  ana  in  some 
of  the  remaining  cases,  establish  design  tolerances  which 
will  allow  for  relatively  easy  qualitative  tweaking  to 
achieve  the  required  fidelity.  There  may  still  be  some 
areas  where  the  data  are  of  little  use,  but  we  can  at 
least  recognize  these  and  avoid  surprises. 

The  data  base  becomes  the  support  structure  tor  all 
subsequent  design  and  test  actions.  It  must  be  as 
complete  as  possible.  Where  holes  exist,  they  shoula  be 
filled  by  such  actions  as  further  data  search,  actual 
tests,  or  estimates  in  collaboration  with  government 
representatives.  If  data  are  inconsistent,  tne  contractor 
should,  through  analysis,  resolve  the  inconsistencies  or 
seek  government  resolution  if  judgment  must  be  exer¬ 
cised.  The  completeness  arx.  accuracy  ot  the  data  base 
cannot  be  overemphasized  since  it  will  serve  as  the 
necessary  resource  for  both  design  and  testing. 

Design 

The  design  must  be  traceable  to  the  data  base. 
Where  data  are  precise  with  little  variation,  the  design 
can  be  accomplished  with  little  allowance  tor  tolerances 
or  tweaking.  If  data  so  indicate,  the  design  should  be 
accomplished  in  such  a  fashion  as  to  permit  significant 
performance  adjustment  without  total  redesign,  txper- 
ience  ond  technical  insight  into  simulator  performance 
has  shown  that  there  ore  three  categories  of  aata  which 
require  different  design  approaches: 
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-  firm,  precise  data  to  support  design 

-  suspect  data  with  large  spread 


] 


-  extrapolated  or  estimated  data  in  substitution  for 
actual  measurements 

Design  and  test  to  the  first  category  of  data  is 
straightforward.  Data  with  large  spread  requires  a  more 
cautious  approach  to  allow  for  adjustments  in  order  to 
meet  required  performance.  Estimated  data  derived  to 
fill  holes  in  the  data  base  provide  design  guidance  but 
obviously  place  the  designer  in  a  risk  position  with 
respect  to  meeting  acceptable  performance.  Creative 
approaches  to  allow  easy  changes  are  in  order  to  reduce 
the  risk. 

Qualification  Test  Procedures  (QTP) 

The  QTP  should  be  prepared  in  parallel  with  the 
design  effort  and  should  be  closely  correlated  with  it. 
When  a  design  is  accomplished  to  meet  a  given  para¬ 
meter  or  performance  curve,  the  testing  procedure  to 
prove  or  confirm  that  the  design  is  correct  should  be 
accomplished  also.  Note  the  very  important  point  that 
the  design  and  the  testing  should  both  be  related  to  a 
common  baseline  contained  in  the  approved  data  base. 
Therefore,  the  ultimate  success  criteria  to  be  used 
during  subsequent  testing,  including  qualification  test, 
are  the  same  as  the  original  design  objectives  and  the 
loop  is  closed.  Of  course,  it  is  not  always  that  easy. 
There  may  be  many  interim  steps  in  both  design  and 
testing  before  a  primary  performance  criterion  cont¬ 
ained  in  the  data  base  can  be  achieved.  Furthermore, 
the  interaction  of  parameters,  particularly  in  the  total 
realm  of  aerodynamic  performance,  can  make  both  the 
design  and  the  testing  a  very  complex  problem.  Never¬ 
theless,  in  order  to  get  a  handle  on  the  qualification  test 
process  we  must  quantify  to  the  extent  possible  the  test 
goals  and  the  logical  source  for  that  quantification  is  the 
data  base  which  serves  as  a  foundation  for  the  design. 

One  further  point  needs  to  be  stressed.  A  poorly 
prepcred  QTP,  produced  at  the  last  minute  in  isolation 
from  design,  invariably  has  a  negative  impact  on  test. 
The  orocedures  simply  don't  work  and  the  goals,  obj¬ 
ectives,  and  success  criteria  are  not  correlated  with  the 
design  and  data  base  so  that  orderly,  effective  testing 
can  be  accomplished.  As  a  result,  the  test  team  is 
forced  to  set  aside  the  official  plan  and  create  a  new 
plan  which  is  workable  and  will  yield  the  desired  results. 
This  slow,  laborious  process  is  wasteful  of  valuable  test 
time.  In  addition,  there  is  a  tendency  to  substitute 
subjective  success  criteria  for  the  objective  criteria 
which  are  contained  in  the  data  base.  In  other  words,  a 
poorly  prepared  QTP  increases  the  risk  during  test  for 
both  the  contractor  and  the  government. 

Test  Readiness 

Government  and  industry  collectively  have  done  a 
poor  job  in  determining  readiness  to  start  qualification 
testing.  Perhaps  our  different  persepectives  have  led  to 
different  expectations  from  the  qualification  testing. 
Industry  may  be  primarily  driven  by  cost  and  schedule 
while  still  wishing  to  field  a  simulator  of  acceptable 
performance.  Gove,  nment,  on  the  other  hand,  places 
simulator  performance  at  the  top,  with  cost  and  schedule 
a  close  second.  These  different  priorities  are  more 
understandable  when  we  recognize  that  our  current  con¬ 
tract  structures  lead  to  limitation  of  government  liabil¬ 
ity  to  contract  ceiling  or  price  (FPIF  or  FFP).  Further¬ 
more,  the  government  acquisition  agencies  have  firm 


commitments  to  the  operational  units  to  provide  high 
quality  simulators  which  meet  original  requirements. 
Anything  less  will  likely  fail  to  be  used  to  full  advantage 
and  the  result  will  be  a  waste  of  taxpayers  money. 

When  we  enter  qualification  testing  the  government 
representatives  expect  to  test  o  machine  which  is  pre¬ 
pared  to  be  put  through  its  entire  operating  envelope  to 
demonstrate  full  compliance  with  the  specification.  This 
testing  should  require  few  changes  or  adjustments  to  the 
system  during  the  tests  and  these  should  largely  be  a 
result  of  the  minor  adjustments  or  tweaking  required  to 
make  the  simulator  respond  correctly  in  the  more  sub¬ 
jective  areas  where  quantitative  data  are  unavailable  or 
inadequate.  If  this  readiness  goal  is  to  be  achieved,  we 
must  make  several  changes  in  the  way  we  currently  do 
business: 

A  complete,  well-defined  design  and  test  baseline 
must  be  established 

-  Discipline  must  be  exercised  to  relate  actions  to 
the  baseline 

-  The  contractor  must  accomplish  his  own  tests 
before  government  qualification  testing  in  order  to  sort 
out  the  problems  and  make  fixes  to  bring  the  simulator 
into  compliance  with  the  specification. 

This  approach  does  not  mean  that  government  ex¬ 
perts  will  not  be  available  to  assist  where  they  are 
needed  in  preparing  the  system  for  qualification  testing. 
Recent  contracts  have  committed  government  testers  to 
provide  early  assistance  in  evaluating  the  simulator 
performance,  particularly  in  the  realm  of  aerodynamics. 
It  is  our  impression  that  this  government  assistance  has 
been  well  received  and  is  valuable  in  catching  problems 
at  an  early  stage.  The  use  of  on-site  government 
representatives  (DCAS  or  similar)  to  go  through  the  QTP 
with  the  contractor  can  be  much  more  effective  than  it 
is  now  if  QTPs  are  better  prepared  and  provide  a  better 
measure  of  system  readiness. 

What  must  be  done  if  we  are  to  avoid  the  test-fix- 
retest  syndrome  is  for  contractors  and  government  to 
make  a  commitment  to  use  the  qualification  test  period 
as  it  was  intended  to  be  used— a  time  to  make  the 
thorough  tests  to  assure  that  contract  specifications  are 
met,  rather  than  a  time  to  make  the  system  meet  the 
specifications. 

Qualification  Test  Period 

Assuming  thot  things  have  been  done  correctly  up  to 
this  point  in  the  project,  the  qualification  test  period 
should  be  anticlimactic.  The  major  system  problems  will 
have  been  sorted  out  and  corrected  during  the  contractor 
and  DCAS  testing  periods.  All  that  is  left  to  be  done  is 
thorough  confirmation  by  the  Air  Force  test  teams  that 
the  system  is  in  compliance  with  the  data  baseline  and  a 
final  tweaking  period  to  make  minor  adjustments  in 
order  to  satisfy  those  subjective  feel  judgments  which 
could  not  be  quantified  or  accurately  predicted  by  the 
government/contractor  technical  team. 

As  a  check  on  test  readiness  it  is  suggested  that  a 
functional  check  flight  (FCF)  be  designed  and  included  in 
the  QTP  to  provide  a  quick  overall  system  check  prior  to 
starting  the  detailed  test  procedures.  This  FCF  would  be 
accomplished  after  the  software  cold  start  is  complete. 
Should  major  problems  still  be  evident,  the  contractor 
would  be  given  time  to  make  corrections  before  storting 
the  detailed  si£system  tests. 


After  tests  are  completed  to  confirm  thot  the  system 
meets  the  QTP  success  criteria  as  related  to  the  original 
data  base,  the  period  of  subjective  testing  and  adjust¬ 
ment  would  begin.  It  is  at  this  point  that  it  may  be 
argued  that  we  should  depart  from  the  traditional  way  of 
doing  business  and  find  some  means  of  more  equitably 
sharing  the  risk  associated  with  making  the  system  fly 
like  the  aircraft.  At  one  extreme  is  the  position  that  the 
contractor  has  no  way  of  predicting  or  controlling  the 
actions  which  will  be  required  by  the  government  to 
accomplish  the  required  performance.  At  the  other 
extreme  is  the  argument  that  the  contractor  through 
experience  and  insight  into  the  necessary  simulator  per¬ 
formance  and  through  assessment  of  the  data  base  and 
requirements  should  be  able  to  predict  the  extent  of 
necessary  tweaking. 

We  submit  that  the  real  world  lives  somewhere 
between  these  extremes.  There  is  tittle  doubt  that  a 
capricious  and  arbitrary  government  test  team,  should 
such  a  team  exist,  can  drive  subjective  testing  for  an 
unnecessarily  long  period  of  time  and  run  contractors 
costs  beyond  expectations.  An  inadequate  data  base  can, 
of  course,  load  the  testing  with  subjective  criteria  which 
can  lead  to  the  same  result.  The  current  constraints  on 
such  activity  are  the  contractor's  reluctance  to  cooper¬ 
ate  and  the  need  by  the  government  to  get  the  system 
into  the  training  inventory.  On  the  other  hand  should  the 
government  agree  to  accept  all  responsibility  for  the 
subjective  test  period,  the  contractor  would  have  little 
motivation  to  take  the  necessary  steps  through  data 
collection/analysis  and  heads-up  design  to  avoid  an  ex¬ 
tended  period  of  subjective  testing  and  tweaking.  There 
might  even  be  a  contractor  tendency  to  unload  activity 
into  the  subjective  test  area  rather  than  expend  the 
effort  to  avoid  it.  It  should  be  noted  that  even  under  the 
full  contractor  risk  scenerio  currently  in  effect,  it  can 
be  argued  thal  some  contractors  are  not  making  a 
maximum  effort  to  stay  away  from  subjective  tweaking. 

Contracting  Arrangements 

What  then  is  the  answer?  How  can  we  structure  Our 
contracts  to  more  equitably  share  the  risks  associated 
with  simulator  development  which  culminates  in  the 
qualification  testing  period? 

-Maybe  a  cost  type  of  contract  for  the  subjective 
test  period  is  more  appropriate  in  light  of  the  risks 
involved. 

-Perhaps  better  management  of  risks  could  lead  to 
reduction  or  elimination  of  them  to  the  point  where  FFP 
is  reasonable. 

-Maybe  it  is  simply  better  utilization  of  FPIF  type 
contracts  which,  after  all,  allow  for  the  sharing  of  risks. 

-Maybe  a  definition  phase  is  required  to  allow  for 
data  collection  and  analysis. 

-Perhaps  the  FPIS  contract  can  be  applied  to  provide 
a  flexible  technique  in  more  closely  aligning  responsi¬ 
bilities  and  risks. 

Let's  briefly  look  at  each  of  these  alternatives  in  our 
search  for  a  better  way. 

A  cost  type  of  contract  for  the  subjective  tweaking 
period  does  not  seem  appropriate  because  it  ploces  all  of 
the  risk  on  the  government  and  does  not  provide  any 
incentive  for  the  contractor  to  minimize  subjective 
tweaking.  In  fact,  it  would  provide  an  incentive  for  the 


contractor  to  maximize  his  profit  unaer  a  FPIF  or  FFP 
basic  contract  and  unload  the  other  ettort  into  tne  suo- 
jective  test  part  of  the  project.  This  approach  woula  oe 
difficult  to  control  from  the  government  point  ot  view. 

If  we  have  a  firm  and  complete  data  baseline  to  serve 
as  a  foundation  for  our  entire  design  ana  test  activities, 
there  is  little  risk  involved  and  contracting  guidance 
would  drive  us  toward  a  FFP  arrangement,  our  exper¬ 
ience  to  date  has  shown  that  complete  and  accurate  aato 
do  not  exist  for  those  simulators  which  we  have  built. 
However,  it  is  our  opinion  that  we  have  tailed  to  maxe 
optimum  use  of  the  data  which  has  been  available  to  us. 
If  we  had  done  so,  the  risks  on  past  ana  existing 
programs  would  have  been  reduced.  In  the  future,  it  is 
possible  that  careful  planning  with  the  acquisition 
agency  for  a  new  aircraft  might  allow  us  to  obtain  most 
of  the  required  data  which  could  serve  as  a  foundation 
for  a  FFP  approach.  It  is  probably  unrealistic  to  totally 
eliminate  all  risk  associated  with  subjective  testing,  but 
the  risk  might  be  minimized  to  the  point  wnere  FFP  is 
reasonable.  This  approach  is  at  best  in  the  future  ana 
may  never  be  feasible  if  concurrency  ot  tne  simulator 
and  the  aircraft  continues  to  be  a  goal.  Nevertheless, 
better  use  of  current  data  will  reduce  risk  out  not 
eliminate  the  problems  associated  with  our  present  con¬ 
tracting  structures. 

Better  utilization  of  our  FPIF  type  contracts  may 
help  resolve  the  problem.  Present  experience,  ot  course, 
indicates  that  most  contracts  go  to  ceiling  ana  contr¬ 
actor's  costs  actually  exceed  ceiling.  Obviously,  the 
sharing  of  costs  and  risks  between  target  ana  ceiling  is 
not  providing  the  cushion  required  in  our  current  way  ot 
doing  business.  Perhaps  our  negotiated  taryets  are  too 
low,  or  maybe  we  are  simply  doing  a  poor  job  ot  manage¬ 
ment.  Competition  has  doubtless  forced  many  in  indus¬ 
try  to  ignore  risks  and  go  against  their  own  better 
judgment  in  making  BAFOs  (Best  And  Final  utters;. 
Assuming  the  subjective  tweaking  is  the  unknown  risk 
which  is  being  suppressed,  we  hove  considered  defininy 
and  setting  aside  this  period  of  testing  in  the  early  stage 
of  the  contract  and  seeking  to  establish  an  equitable 
share  ratio  over  and  beyond  the  basic  contract  should  the 
ceiling  be  exceeded.  For  example  at  Pl)K  or  UJK 
(Preliminary  or  Critical  Design  Keviewf  we  would  have 
an  in-depth  review  of  the  date  base/analysis  and  identity 
those  parameters  which  would  likely  be  siAject  to  qual¬ 
itative  tweaking.  With  this  knowledge  we  would  nego¬ 
tiate  some  share  ratio  based  on  our  risk  assessment 
which  would  come  into  effect  only  if  the  ceiling  of  the 
basic  contract  were  exceeded.  This  share  ratio  would 
place  more  burden  on  the  contractor  than  the  basic 
contract  share  ratio  in  order  to  encourage  completion  ot 
the  project  within  ceiling  and  discourage  the  shitting  ot 
work  from  the  basic  contract  to  the  above  ceiling 
sharing  arrangement. 

There  are  some  problems  with  this  proposal: 

-It's  open  ended 

-Government  budgeting  would  be  difficult 

-If  may  be  tough  to  reach  agreement  on  the  above 
ceiling  share  ratio 

-We  would  have  to  find  some  way  to  identify  and 
contain  the  basic  low  risk  work  within  the  primary  FPIF 
contract 

Becouse  of  the  above  problems,  the  contracting 
community  does  not  support  this  approach. 
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It  has  been  suggested  that  the  big  unknown  facing 
industry  at  the  time  of  proposal  and  negotiation  is  the 
availability  of  complete  and  accurate  data  an  which  to 
base  the  simulator  design  and  testing.  Furthermore, 
because  of  the  magnitude  of  the  task  to  collect  and 
analyze  the  data,  contractors  have  neither  the  time  or 
the  resources  to  resolve  this  unknown  during  the  proposal 
phase.  In  order  to  reduce  the  risk  associated  with  the 
data  base,  it  has  been  suggested  that  a  definition  phase 
contract  be  awarded.  This  approach  would  remove  much 
of  the  risk,  but  it  would  also  extend  this  time  required  to 
provide  a  training  device  to  the  field.  Since  the  time 
delay  between  definition  of  requirements  and  ready-for- 
training  status  is  already  considered  much  too  long,  this 
approach  cannot  be  recommended. 

Another  way  of  contractually  recognizing  the  prob¬ 
lem  of  the  data  base  availability  aid  content,  along  with 
its  impact  on  testing,  is  through  the  use  of  a  FPIS 
arrangement.  This  contracting  technique  allows  the 
target  to  be  reset  at  some  well  defined  contract  mile¬ 
stone.  In  the  case  of  the  simulator,  we  could  define  the 
reset  to  occur  at  PDR  or  CDR,  a  point  in  the  program 
where  we  should  have  the  data  base  problem  under 
control  and,  therefore,  have  a  much  better  idea  of  the 
subjective  tweaking  required  to  achieve  our  fidelity 
objective.  Although  ceiling  and  share  ratio  would  remain 
the  same,  the  adjustment  of  target  would  allow  full 
recoupment  of  some  costs  and  reduce  the  risk  to  the 
contractor  at  a  point  when  those  risks  are  more  fully 
defined.  This  FPIS  approach  is  recommended  as  the 
most  appropriate  method  to  more  closely  align  the  risks 
and  responsibilities  involved  in  the  simulator  acquisition 
process. 


Mr  David  P.  Glenn  is  the  Assistant 
Director,  Deputy  for  Simulators, 
Aeronautical  Systems  Division  at  Wrighf- 
Patterson  AFB  OH.  In  this  position,  Mr 
Glenn  assists  in  the  overall  management  of 
Air  Force  simulator  development  and 
acquisition.  Prior  to  this  assignment  Mr 
Glenn  spent  13  years  as  an  engineer  and 
program  manoger  involved  in  missile  and 
space  vehicle  testing.  He  holds  masters 
degrees  in  physics  and  business 
administration. 

Major  Lester  H.  Baer  is  currently  the 
Director  of  Test  and  Deployment  for  the 
Deputy  for  Simulators.  His  directorate 
manages  the  testing  of  all  Air  Force  air¬ 
crew  training  devices,  which  presently  in¬ 
cludes  the  F-16,  A- 10,  B-52,  C-130,  C- 141 
and  C-5.  He  was  formerly  the  Tactical  Air 
Command  project  officer  for  the  OT&E  of 
the  F-15  flight  simulator.  He  received  a 
B.S.  in  Engineering  Science  from  the  U.S. 
Air  Force  Academy  and  holds  a  M.S.  in 
Education  (Instructional  Technology)  from 
the  University  of  Southern  California.  He 
is  a  command  pilot  with  over  2700  flying 
hours  in  nine  aircraft  types. 


SUMMARY 

The  historical  record  end  our  own  experience  indi¬ 
cates  that  qualification  testing  is  requiring  much  more 
time  than  we  include  in  our  program  plans.  This  ex¬ 
tension  of  test  time  impacts  both  contractor  costs  and 
system  deployment.  The  problem  of  test  extension  is  not 
limited  just  to  the  test  period  but  permeates  the  entire 
program  starting  with  requirements  definition.  Our  look 
at  the  principle  elements  of  requirements,  data  base, 
design,  qualification  test  procedures,  test  readiness,  and 
actual  tests,  as  these  relate  to  the  problem,  has  led  to  a 
number  of  recommendations  to  improve  our  performance 
and  reduce  the  risks  for  both  government  and  industry. 
The  importance  of  the  da  to  base  as  a  foundation  for 
subsequent  design  and  testing  activity  has  been  stressed. 
We  have  examined  the  risks  and  concluded  that  they  can 
be  reduced  by  better  government/ contractor  manage¬ 
ment  of  the  program  aid  through  more  suitable  contr¬ 
acting  arrangements.  Specifically,  it  is  recommended 
that  an  FPIS  approach  be  used  with  reset  at  the  PDR  or 
CDR  milestones  in  order  to  make  allowances  for  un¬ 
knowns  relative  to  the  date  base  and  the  effects  of  these 
on  qualification  testing  schedules. 

The  recommendations  contained  herein  will  not  serve 
as  a  panacea  for  all  the  problems  we  jointly  face  in 
trying  to  provide  the  best  possible  aircrew  simulators  to 
our  operating  forces,  but  it  is  hoped  that  these  ideas  will 
serve  as  a  catalyst  for  further  improvements  in  the  way 
we  do  business,  particularly  as  they  relate  to  the  tough 
problem  of  qualification  testing.  Let  us  not  forget  that 
we  have  a  responsibility  to  ourselves  and  this  country  to 
provide  the  means  for  our  aircrews  to  be  the  best  trained 
in  the  world.  This  objective  demands  our  continued 
efforts  to  produce  high  quality  simulators,  which  are 
delivered  on  time  and  within  cost.  Solution  of  the 
qualification  test  problems  will  be  a  giant  step  toward 
this  objective. 
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ABSTRACT 

ON-LINE  CONFIGURATION  MANAGEMENT:  Configuration  Management  is' covered  in  many  books  and 
articles  and  is  described  in  detail  for  development  and  test  of  systems  which  are  either  all 
software  or  all  hardware.  In  real  time  simulation  which  involves  integrated  and,  many  times, 
interactive  hardware  and  software,  plus  human  interaction  with  instructor  and/or  students,  a 
new  level  of  control  is  required  called  hardware  software  integration/test.  In  addition, 
subjective  customer  evaluation  and  acceptance  may  extend  this  period  indefinitely.  This 
period  of  test  and  evaluation  could  result  in  elapsed  time  of  half  the  project  schedule  and 
is  usually  where  all  the  budgets/schedules  go  down  the  tubes.  This  paper  describes  an 
approach  to  on-line  system  configuration  management,  starting  with  a  prototype  baseline 
system  ready  for  hardware  software  integration. 


INTRODUCTION 

An  Indus try/ Government  meeting  held  on 
November  26,  1979  prior  to  the  first  interservice/ 
industry  training  conference  concluded  with  a  list 
of  initial  planning  and  acquisition  issues  for  a 
panel  discussion  during  the  conference.  The  pur¬ 
pose  was  to  gain  a  better  understanding  of  problem 
areas,  to  identify  and  discuss  probable  causes, 
and  to  reach  some  agreement  on  possible  recommenda¬ 
tion  to  improve  performance.  Under  the  topic 
"Contractor  Performance,"  the  following  problem 
areas  were  described: 


"The  need  exists  for  the  development  of  more 
definitive  software  management  plans  with  mile¬ 
stones  adequate  for  monitoring  developmental  prog¬ 
ress.  Concomitantly,  determinative  plans  for  the 
integration  of  software  with  hardware  is  required. 
Frequently  either  software  development  lags  and/ 
or  extensive  integration  problems  are  encountered; 
the  resultant  effort  to  catch  up  or  correct  causes 
delays  and  additional  costs." 

"Practically  every  major  training  device  pro¬ 
gram  has  experienced  difficulty  in  the  development 
of  software  and  its  integration  with  hardware. 
Underestimating  software  development  adversely  im¬ 
pacts  time,  schedule,  and  performance  requirements, 
and  creates  serious  life  cycle  support  problems. 
What  can  be  done  by  contractor  management  to  place 
the  required  emphasis  on  software  development?" 

"Status  reporting  systems  are  only  creditable 
if  they  factually  state  progress  on  the  program 
and  highlight  critical  issues  in  management  or 
engineering  that  require  decisions.  What  can  be 
done  to  improve  the  quality  and  timeliness  of  sta¬ 
tus  Information  provided  by  contractors?" 

This  paper  addresses  these  problem  areas,  par¬ 
ticularly  the  last  paragraph  on  reporting  systems, 
as  they  occurred  during  the  development  of  the  T- 
44  Ope  ional  Flight  Trainer  (OFT).  The  T-44  OFT 
develo:  nt  program  was  the  first  project  which 
applieu  the  on-line  configuration  management 
technique. 


DEFINITIONS 

Reference  Figure  1  for  pictorial  presentation. 

1.  On-Line  -  The  complete  hardware/software 
system  under  operation  and  test  in  real  time. 

2.  Verification  -  Test  the  computer  program 
to  ensure  that  it  satisfies  all  performance  and  de¬ 
sign  requirements;  e.g.,  software  engineering  tests. 

3.  Validation  -  Test  the  computer  program 
to  ensure  that  it  operates  properly  in  the  system 
environment  and  that  it  satisfies  all  system 
requirements;  e.g.,  system  engineering  tests. 

4.  Hardware  Software  System  Integration  - 
This  type  of  testing  executes  the  computer  program 
in  a  completely  real  (on-line)  environment  utiliz¬ 
ing  actual  hardware.  This  test  demonstrates  that 
the  computer  program  satisfies  all  requirements  of 
the  program  performance  specification  while  being 
executed  in  a  real  system  environment. 

5.  Acceptance  Test  Procedures  (ATP)  or  Cer¬ 
tification  -  This  type  of  testing  makes  use  of  the 
computer  program  after  it  has  completed  system 
hardware  software  integration.  This  test  is  in¬ 
tended  to  exercise  all  subprograms  of  the  computer 
program  in  consonance  with  the  hardware.  These 
tests  certify  that  all  software,  hardware,  and 
system  requirements  have  been  satisfied.  These 
tests  are  defined  in  the  Acceptance  Test  Proced¬ 
ures  document.  Figure  1  shows  the  relationship 

of  verification,  validation,  and  certification 
testing. 

CONFIGURATION  MANAGEMENT 

Configuration  management  (CM)  is  a  discipline 
traditionally  applied  to  the  development  of  hard¬ 
ware  systems  or  to  the  development  of  hardware  ele¬ 
ments  of  hardware  software  systems.  Its  applica¬ 
tion  in  connection  with  other  disciplines  leads  to 
orderly  and  structured  system  development.  CM  is 
generally  concerned  with  the  consistent  labeling, 
tracking,  and  change  control  of  the  hardware  ele¬ 
ments  of  a  system.  Software  configuration  manage- 
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PROGRAM 

MAINTENANCE 


Figure  1.  Verification,  Validation,  and  Certification 


ment  is  CM  tailored  to  a  system,  or  portion  of  a 
system,  predominantly  comprising  software  or 
firmware. 

Figure  2  depicts  the  stages  of  the  system 
life  cycle.  Although  each  stage  is  equally  ap¬ 
plicable  to  hardware  and  software,  the  differ¬ 
ences  between  hardware  and  software  development 
and  production  give  rise  to  different  specific 
requirements  for  managing  the  software  life  cycle. 
Each  stage  of  development  culminates  in  a  base¬ 
line,  (Figure  3)  a  point  at  which  management  has 
the  opportunity  to  view  the  system  in  detail  in 
order  to  examine  it;  integrity  -  that  is,  its 
adherence  to  and  satisfaction  of  the  operational 
requirements. 


After  both  hardware  and  software  become 
independently  operational,  a  period  of  hardware 
software  integration  (HSI)  begins  and  continues 
until  all  acceptance  testing  is  completed  and 
the  device  validated.  The  basic  four  components 
of  configuration  management  will  be  addressed 
during  HSI  and  acceptance  testing  as  depicted  in 
Figure  4. 

Configuration  Identification 

Identification  during  hardware  software 
integration  and  acceptance  testing  is  well 
defined  and  takes  place  at  the  subprogram  level. 
A  subprogram  is  a  subdivision  of  a  computer 
program  that  accomplishes  an  entire  set  of 
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Figure  3.  Computer  Software  Life  Cycle 


WHAT  IS  MY  SYSTEM  CONE IGURATION?— ♦ 

HOW  DO  I  CONTROL  CHANGES  TO  MY  CON- _ 

FIGURATION?  — *■ 

WHAT  CHANGES  HAVE  I  MADE  TO  MY  _  r 
SYSTEM? 

DOES  THE  SYSTEM  I  AM  BUILDING  SATISFY-^- 
THE  PERFORMANCE  REQUIREMENTS? 


CONFIGURATION 

MANAGEMENT 


IDENTIFICATION 


CONTROL 


YOUR  SYSTEM  CONFIGURATION  CONSISTS  OF  THE  FOLLOWING 
ITEMS:  (ITEMl  ITEM, . 1TEMJ 

Y  THE  STEPS  IN  PROCESSING  CHANGES  THAT  DIRECTLY  OR  IN¬ 
DIRECTLY  AFFECT  YOUR  CONFIGURATION  ARE:  (STEP,,  STEP; 
....  STEP  ) 


STATUS  ACCOUNTING 


AUOITING 


YOUR  SYSTEM  CONFIGURATION  AND  RELATED  CHANGES  AT  THIS 

"TIME  ARE:  (ITEM,,  ITEM, . ITEM  )  +  (CHANGE,,  CHANGE,, 

CHANGE,,  PENDING  CHANGE,,  ...7  PENDING  CHANGE^) 

YOUR  SYSTEM  AS  CURRENTLY  BUILT  DIFFERS  FROM  THE  REQUIREMENTS 
AND  DERIVATIVE  SPECIFICATIONS  AS  FOLLOWS:  (DIFFERENCE,, 
DIFFERENCE, . DIFFERENCE  ) 


Figure  4.  The  Four  Components  of  Configuration  Management 


program  performance  requirements;  e.g.,  fuel 
system.  Therefore,  it  is  a  specific  testable 
entity. 


The  tests  are  also  well  defined.  A  complete 
and  approved  Acceptance  Test  Procedure  is  utilized 
(and  proofed)  during  this  period  of  configuration 
management.  Note  that  during  all  the  testing  we 
are  asking  the  question,  does  the  system  satisfy 
the  performance  requirements?  The  whole  testing 
philosophy  is  keyed  to  the  audit  concept  of  dis¬ 
crepancies;  e.g.  "The  current  system  differs  from 
the  requirement  as  follows;1'  The  audit  is  con¬ 
tinuous.  The  schedule  is  keyed  to  the  audit. 


On  the  T-44  OFT,  the  schedule  was  developed 
by  the  project  engineer  and  based  on  the  estimates 
to  complete  obtained  from  his  engineering  staff, 
the  status  of  the  hardware,  and  the  subprogram 
interdependencies.  Each  subprogram  was  identified 
in  accordance  with  the  approved  acceptance  test 
document  and  organized  in  schedule  format  for 
each  of  two  shifts.  Included  on  the  schedule  was 
a  weekly  summary  status  accounting  of  discrepancy 
reports  (DRs).  Figure  5  indicates  graphically 
normal  DR  generation  and  abnormal  DR  generation. 
Note  that  a  normal  curve  peaks  out  and  drops  off 
towards  zero.  A  glance  at  the  schedule  (Figures 
6,  7,  8,  9)  provides  the  observer  with  a  top  level 
audit  and  trend.  For  example  "Are  more  DR's  being 
generated  than  fixed?"  Having  established  what 
the  configuration  is  and  a  method  of  determination 
of  status,  we  will  now  look  into  methods  of 
control . 


Configuration  Control 

Controls  take  two  significant  forms;  the  first 
is  control  of  changes  to  the  software  configuration 
and  the  second  is  control  of  the  status  of  testing. 
On  the  T-44  OFT  the  problem  was  compounded  some¬ 
what  by  having  two  devices  (trainers)  entering  HSI 
at  approximately  the  same  period  in  time.  There¬ 
fore  the  following  management  procedure  was 
established. 

Software  Configuration  Control.  Every  Friday 
morning  prior  to  the  0800  shift,  system  1  was 


backed  up  on  magnetic  tape.  System  2  was  made 
identical  to  system  1  by  restoring  system  1  backup 
tape. 

System  1  was  considered  the  master  system.  All 
other  systems  were  considered  as  test  beds  for 
changes.  It  was  the  responsibility  of  each  soft¬ 
ware  engineer  to  update  system  1  with  any  modified 
models  or  programs  by  informing  the  librarian  of 
the  files  to  be  moved  to  system  1  and  on  which 
system  they  currently  resided. 

All  system  discrepancies  (hardware  and  soft¬ 
ware)  were  to  be  reported  using  the  Discrepancy 
Report  (DR)  form  (Figure  10).  The  filled  in  form 
was  given  to  the  librarian  who  was  responsible  for 
assigning  a  DR  number  and  entering  it  in  the  DR 
master  log.  Hardware  DR's  were  then  passed  on  to 
systems  personnel  for  handling.  As  each  DR  was 
corrected,  an  entry  to  that  effect  was  made  in 
the  master  log.  The  DR  was  then  defined  as 
"TESTED".  A  DR  could  only  be  defined  as  "CLOSED" 
when  tested  and  signed  off  by  the  project 
engineer. 

As  each  DR  was  tested  and  closed,  the  backup 
tape  number  was  recorded  to  provide  traceability 
for  all  changes.  All  changes  were  in  the  form  of 
source  code  updates.  No  patches! 

A  job  accounting  system  incorporated  into  the 
vendor's  computer  operating  system  was  used  as  a 
general  accounting  ledger  to  track  all  changes. 

The  accounting  file  was  listed  every  Friday  morning 
and  also  saved  on  the  system  backup  tape.  After 
the  file  had  been  saved,  it  was  purged  and  made 
ready  for  the  next  week.  The  job  accounting  list¬ 
ing,  the  backup  tape  number,  and  the  DR  masterlog 
provided  an  excellent  means  of  managing  the  prog¬ 
ress  of  the  project  and  controlling  the  configura¬ 
tion  of  the  system. 

Testing  Status  Control.  System  test  control 
was  established  by  setting  up  good  communications 
and  by  test  status  documentation. 

Communication: 

Established  weekly  all  hands  meeting. 

Wrote  minutes  for  review. 

Determined  significant  problems. 

Established  responsibilities  and  action 
plans. 
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□  DR  CUM  STATUS 


Figure  6.  T-44  System  Integration  and  Evaluation  Shift  1  (Month 
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□  DR  CUM  STATUS 

Figure  7.  T-44  System  Integration  and  Evaluation  Schedule  Shift  2  (Month  1) 


COMMUNICATIONS 

AURAL  CUE 

CONTROL  LOADING 

INSTRUCTOR  STATION 

PITCH  ON  GROUND 

ENGINE  FLIGHT  (2) 

SURFACE  VS  CONTROL 
DEFLECTION 
PROPELLER  THRUST 

CONTROL  FRICTION  &  INERTIA 

TRIM 

LONGITUDINAL  STABILITY 
LATERAL  STABILITY 
SPIRAL  STABILITY 


Q  DR  CUM  STATUS 


Figure  8.  T-44  System  Integration  and  Evaluation  Shift  1  (Month  2) 
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Figure  9,  T-44  System  Integration  and  Evaluation  Shift  2  (Month  2) 


DISCREPANCY  REPORT  (DR) 

PROJECT: 

DATE: 

ISSUER: 

PROBLEM 

NO.  : 

RESP.  ENG.: 

REF: 

TYPE  OF  PROBLEM 

(  )  HARDWARE: 

SYSTEM  #  (  )  SOFTWARE: 

SYSTEM 

STATEMENT  OF  PROBLEM 

CORRECTIVE  ACTION  TAKEN 

TESTED 

DATE 

TAPE  « 

ENGINEER 

DATE 

TAPE  # 

ENGINEER 

COMMENTS 

Figure  10.  Discrepancy  Report  Form 
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Accounting: 

Set  up  test  schedule  keyed  to 
discrepancies. 

Updated  discrepancy  form. 

Librarian  had  full  authority  and 
responsibility  for  accounting. 

Job  accounting  system  incorporated 
into  the  operating  system. 

The  status  accounting  report,  described 
below,  was  utilized  at  the  weekly  meetings  and 
proved  to  be  the  key  ingredient  to  successful 
communication.  Each  person  knew  his  responsi¬ 
bility  and  management  visibility  allowed  the 
decision-making  processes  to  work! 

Status  Accounting 

In  order  to  keep  track  of  all  hardware  and 
software  problems  which  occur  during  Hardware 
oftware  Integration  (HSI)  or  the  Acceptance 
esting  Procedure  (ATP),  a  Discrepancy  Report 
(DR)  is  filled  out.  As  each  discrepancy  is 
reported,  a  control  number  is  assigned  by  the 
librarian  and  entered  in  the  master  log.  The 
function  of  this  log  is  to  keep  a  record  of  all 
DRs  and  present  this  information  in  various 
ways. 

The  DR  Status  Accounting  Report,  written 
in  FORTRAN  IV,  will  operate  on  the  project 
computer  system  in  the  interactive  mode.  This 
interactive  program  reads  a  master  log  entry  in 
card  image  format,  processes  the  input  data,  and 
prints  a  hardcopy  output  report  as  specified 
below. 

The  program  can  be  applied  to  any  trainer 
project  by  preparing  an  appropriate  set  of  inputs. 
The  program  input  file  (project  data  base)  can  be 
divided  into  two  sections.  The  first  part  (sys¬ 
tems  section)  describes  the  major  systems  of  the 
trainer  and  generally  correlates  with  the  computer 
subprograms.  The  second  part  (data  section)  of  the 


file  represents  the  DR  data  base.  It  is  informa¬ 
tion  that  is  taken  from  the  DR  log  and  added  to  the 
data  base  periodically.  Thus  the  systems  data  re¬ 
mains  fixed  for  a  given  project  while  the  DR  data 
base  increases  as  the  project  progresses. 


The  first  record  of  the  systems  section  con¬ 
tains  the  name  of  the  program  ( i . e . ,  T-44,  C-130, 

2 IE  1 7 ,  etc.).  Next  are  the  system  descriptions, 
one  per  record,  along  with  a  unique  system  letter. 
After  the  system  description  comes  the  data  sec¬ 
tion.  This  information  includes  the  DR  number, 
system  letter(s),  initials  of  the  assigned  engi¬ 
neer,  date  tested  (when  applicable),  failed  testing 
or  waiting  for  parts  indicators,  date  closed  (when 
applicable),  a  brief  description  of  the  problem, 
and  the  number  of  the  backup  tape  on  which  the 
problem  solution  first  appears. 


The  program  reads  the  input  file  and  prepares 
the  master  DR  log  which  contains  the  status  of  each 
DR  Drinted  out  in  numerical  order.  A  summary  table 
is  compiled  which  lists  each  system,  the  number 
of  open,  closed,  or  tested  DRs,  the  total  number 
for  each  system,  and  the  totals  for  the  whole  job. 


The  information  is  then  resorted.  The  first 
resorting  procedure  prints  out  all  DRs  which  are 
not  closed.  The  next  sort  will  print  out  a 
listing  of  all  DR's  which  are  ready  for  testing. 
The  non-closed  DRs  are  then  resorted  again  for  a 
third  and  fourth  time  and  printouts  are  made  by 
system  and  then  by  assigned  engineer. 

Last,  another  table  is  compiled  showing  the 
DR  totals  -  open,  tested,  closed,  and  totals  -  by 
responsible  engineer. 

Figure  11  identifies  the  format  and  contents 
(as  described  above)  and  indicates  that  the  loop 
is  closed  by  summarizing  the  DR’s  by  subprogram 
on  the  schedule. 


1. 

FORMAT: 

DR.  SYSTEM  RESP  DATE  DATE 

NUMBER  AFFECTED  ENG  TESTED  CLOSED 

DESCRIPTION 

TAPE 

STATUS  NO. 

2. 

CONTENTS: 

• 

SYSTEM  LEGEND  -  IDENTIFICATION 

• 

MASTER  DISCREPANCY  LOG 

-  SYSTEM  SUMMARY  STATISTICS 

• 

OPEN  DISCREPANCY  LOG 

-  READY  FOR  CLOSEOUT  TEST 

• 

STATUS  OF  EACH  DEFINED  SYSTEM 

• 

RESPONSIBILITY  LOG 

-  INDIVIDUAL  RESPONSIBILITY  STATUS 

3. 

UPDATE: 

• 

DAILY 

t 

WEEKLY  -  DISTRIBUTED  AND  REVIEWED 

4. 

SCHEDULE: 

• 

SETUP  LOGICAL  SYSTEM  TEST  SEQUENCE 

• 

KEYED  SCHEDULE  STATUS  TO  DR'S 

• 

TRACK  DR  HISTORY 

Figure  11.  T-44  Job  Accounting 
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Auditing 

The  final  audit  process  follows  naturally 
from  the  HS1  in  that  system  testing  proceeds 
into  Government  acceptance  testing  utilizing  the 
Government-approved  document.  This  occurs  after 
in-house  testing  has  reduced  all  significant 
discrepancies  tc  zero. 

CONCLUSIONS 

The  approach  described  herein  provided 
never-before-attained  management  visibility  into 
audit  status.  AH  parties  were  able  to  review 
detailed  printouts  as  to  the  exact  status  of  the 
project.  Responsibilities  were  clear  and  it  was 
obvious  that  the  job  was  under  control.  The 
printouts  proved  to  be  extremely  valuable  when 
used  as  the  basis  for  communications,  both  at 
in-house  meetings  and  for  the  customer's  review. 

As  a  result  of  this  experience,  Gould,  Inc., 
Simulation  Systems  Division  Policy  and  Standards 
for  "On-Line  Software  Configuration  Management" 


have  been  developed  and  utilized  on  a  number  of 
projects  with  excellent  results. 

Computer  Printout  Examples. 

A  typical  example  of  one  of  our  programs  is 
shown  in  the  following  illustrations.  Figure 
12  summarizes  the  DR  statistics  by  system. 

Figure  13  depicts  the  last  page  of  the  master 
software  DR  log.  Figure  14  summarizes  open  DRs 
by  responsible  engineer.  This  printout  is 
utilized  at  weekly  meetings  as  a  basis  for  re¬ 
view  status.  Figure  15  summarizes  the  DR 
statistics  by  responsible  engineer. 
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SYSTEM 

OPEN 

T44  DR  STATISTICS 

TESTED  CLOSED 

TOTAL 

ENGINE  START 

0 

0 

11 

11 

GROUND  DYNAMICS 

0 

0 

17 

17 

ENGINE  GROUND 

2 

0 

2 

4 

ACCEL,  TO,  STOP  G  DECEL 

0 

0 

1 

1 

FLIGHT  INSTRUMENTS 

1 

0 

9 

10 

WHEELS  G  BRAKES/CRASH 

0 

0 

4 

4 

PROPS 

0 

0 

4 

4 

ATMOSPHERICS/ROUGH  AIR 

0 

0 

2 

2 

ENGINE  FLIGHT 

0 

0 

13 

13 

STALL  AND  BUFFET 

0 

0 

1 

1 

ELECTRICAL  SYSTEM 

2 

0 

34 

36 

FUEL  SYSTEMS 

0 

0 

13 

13 

GEAR  G  FLAPS 

0 

0 

16 

16 

ANNUNCIATOR  G  FIRE  SYSTEM 

0 

0 

0 

0 

ECS  5  MISC  SYSTEMS 

1 

1 

13 

15 

HARDWARE 

23 

0 

126 

149 

DRED 

1 

0 

10 

11 

EXEC  G  DEBUG 

1 

0 

6 

7 

control  loading 

2 

0 

20 

22 

INSTRUCTOR  STATION 

23 

1 

80 

104 

NAVIGATION 

4 

1 

26 

31 

NAV  RECEIVERS 

0 

0 

14 

14 

MOTION 

2 

0 

10 

12 

autopilot 

1 

0 

6 

7 

FLIGHT  G  AERO 

3 

0 

23 

26 

INPUT/OUTPUT 

4 

0 

11 

15 

COMMUNICATIONS 

3 

0 

2 

5 

AURAL  CUE 

0 

0 

2 

2 

PITCH  ON  GROUND 

0 

0 

0 

0 

SURFACE  VS  CONTROL  DEFLT 

0 

0 

0 

0 

PROPELLER  THRUST 

0 

0 

0 

0 

CONTROL-FRICTION/ INERT  I  A 

0 

0 

0 

0 

TRIM 

2 

0 

0 

2 

LONGITUDINAL  STABILITY 

0 

0 

0 

0 

LATERAL  STABILITY 

0 

0 

0 

0 

SPIRAL  STABILITY 

0 

0 

0 

0 

REPLAY 

0 

0 

1 

1 

NCS-31  SELF  TEST 

0 

0 

0 

0 

ATP  VERIFICATION 

0 

0 

0 

0 

INSTRUMENTS 

9 

0 

5 

14 

totals 

84 

3 

482 

569 

Figure  12.  T-44  DR  Statistics  for  Each  System 
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Figure  14.  Open  DR  Log  Keyed  to  Responsible  Engineer 


T  44  DR 

STATISTICS 

ENGINEER 

OPEN 

TESTED 

CLOSED 

TOTAL 

RTM 

7 

0 

36 

43 

JR 

26 

0 

135 

161 

TJD 

9 

1 

96 

106 

SB 

3 

0 

41 

44 

DDD 

5 

0 

19 

24 

TZ 

4 

1 

35 

40 

HB 

3 

0 

5 

8 

GL 

2 

0 

64 

66 

RA 

20 

1 

51 

72 

JM 

5 

0 

0 

5 

TOTALS 

84 

3 

482 

569 

Figure  15.  T-44  DR  Statistics  by  Responsible  Engineer 
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SOFTWARE  DEVELOPMENT  FOR  THE  MULTI-ENVIRONMENT  TRAINER  (MET) 
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San  Diego,  California 

ABSTRACT 

The  MET  system  is  a  tactical  team  trainer  designed  to  simulate  multiple-ship  tactical 
operations.  The  functions  being  simulated  span  the  under  sea,  surface,  and  air  environments  and 
include  operations  in  the  Bridge,  Combat  Information  Center  and  Sonar  areas.  The  MET  training 
system  is  designed  about  the  use  of  eight  computers  configured  in  two  groups  of  four.  Each  group 
of  four  computers  simulates  the  functions  of  a  PCG  (patrol  croft  gunboat)  class  ship. 

The  MET  software  development  will  result  in  the  generation  of  approximately  150,000 
computer  instructions,  a  portion  of  which  reside  in  eoch  of  the  four  computers  which  simulate 
tactical  ship  operations.  Management  control  and  technical  development  of  the  MET  has  been 
characterized  by  the  development  and  adoption  of  meaningful  software  development  practices  and 
procedures  and  adherence  to  the  fundamental  concept  that  software  design  must  precede  coding. 


INTRODUCTION 
System  Description 

The  MET  system  is  a  tactical  team  trainer 
designed  to  simulate  multiple-ship  tactical 
operations.  The  functions  being  simulated  span 
the  under  sea,  surface  and  air  environments  and 
include  operations  in  the  Bridge  (ship  control), 
Combat  Information  Center  (air/surface  search 
radar,  electronic  warfare  and  weapon  fire 
control),  and  Sonar  areas  (Passive/Active  sonar 
and  torpedo  fire  control),  see  Figure  I. 

The  MET  training  system  is  designed  about 
the  use  of  eight  computers  (SEL  32/75) 
configured  in  two  groups  of  four  processors. 
Each  group  of  four  computers  performs  a  set  of 
dedicated  functions,  in  real-time,  which 
simulate  the  operation  of  a  single  PCG  class 
ship.  The  various  modes  of  trainer  operation 
permit  the  execution  of  exercises  involving 
either  one  or  two  ships.  Data  exchange  among 
the  eight  computers  is  accomplished  via  a  block 
of  shared  memory  while  access  to  required 
peripherals  is  controlled  by  a  program  directed 
peripheral  switch,  see  Figure  2. 

Software  Description 

The  MET  software  architecture  is 
characterized  by  five  subsystem  areas:  Master 
Control,  Passive  Sonar,  Active  Sonar,  Radar  and 
Off-Line  Processes.  The  first  four  subsystem 
areas  represent  the  real-time  portion  of  the 
MET  software  which  resides  in  each  of  the  two 
sets  of  four  MET  computers.  The  remaining 
software,  which  does  not  execute  in  real-time,  is 
referred  to  as  the  Off-Line  Processes  and 
includes  such  programs  as  Diagnostics,  Daily 
Readiness  Test,  Data  Base  Generators,  etc. 

Execution  of  the  MET  software,  resident  in 
each  processor,  is  directed  by  a  simulation 
executive  which  operates  under  control  of  the 
Real-Time  Monitor  (RTM)  operating  system. 
Several  aspects  of  MET  software  design 
contribute  toward  making  the  development  of 
that  software  a  challenge.  To  begin  with,  much 
of  the  MET  software  interacts  with  government 
furnished  equipment  (GFE),  performing  the  tasks 
of  stimulation  and  simulation.  Secondly,  a 


significant  interplay  exists  between  the  various 
programs  resident  within  a  given  processor  or 
processors.  Finally  the  values  computed  by  all 
of  the  MET  processors  contribute  to  a  large  and 
reasonably  complex  system  data  base  which 
must  be  accessable  to  all  computers  in  a 
manner  which  permits  the  satisfaction  of  the 
system's  real-time  requirements. 


Development  Concepts 

Several  concepts,  basic  to  the 
development  of  software,  were  adopted  and/or 
augmented  for  use  on  the  MET  project.  These 
concepts  were  selected  with  an  objective  in 
mind  of  defining  a  set  of  procedures  which 
could  be  utilized  throughout  the  company  for 
the  development  of  future  software  systems. 
Each  of  these  basic  concepts  is  discussed 
briefly  below. 

Chief  Programmer  Team.  The  MET 
software  organization  utilizes  the  Chief 
Programmer  methodology.  A  team  of  software 
engineers  led  by  a  chief  programmer  is 
responsible  for  the  design,  documentation, 
implementation  and  integration  of  each 
subsystem  area  of  software  development.  Each 
of  the  chief  programmers  and  several  of  the 
members  of  each  subsystem  team  are 
continuously  involved  from  the  initial  design 
phase  through  implementation  and  test  and 
finally  each  team  contributes  one  or  two 
members  to  a  software/system  integration 
team. 

Unit-Development  Folder  (UDF).  A  unit 
development  folder  is  prepared  for  each  unit  of 
code  developed  and  controlled  using  software 
configuration  control  procedures.  On  the  MET 
project  that  unit  is  a  Computer  Program 
Module  (CPM).  The  unit  development  folder 
serves  as  the  centrol  historical  source  of 
development  information  for  a  given  CPM  and 
the  basis  from  which  contract  deliverable 
documents  are  prepared.  The  unit  development 
folder  contains: 


1.  Requirement  Information  -  What  the 
module  is  to  perform  as  described  in  the  con¬ 
tract  specification,  prepared  in  such  a  manner  as 
to  permit  inclusion  in  the  Program  Performance 
Specification  (PPS)  document  with  little  or  no 
format  modification. 

2.  Design  Information  -  A  description 
of  the  module  design  (control  flow,  data  flow, 
inputs,  output  interaction  with  other  CPMs), 
which  is  included  in  the  Program  Design 
Specification  (PDS)  document  with  little  or  no 
format  change. 

3.  Detailed  Design  Description  -  A 
detailed  implementation  description  which  in¬ 
cludes  a  flowchart  and  narrative  for  the  CPM 
and  each  submodule  or  subroutine  utilized  by  the 
CPM.  The  narrative  is  keyed  to  the  flowchart  by 
the  use  of  labels  which  appear  on  both  the 
flowchart  and  source  code  listing,  insuring  direct 
traceability  from  document  to  source  listing. 

The  detailed  CPM  design  description  is 
prepared  in  a  format  which  permits  direct  in¬ 
corporation  into  the  Program  Description 
Document  (PDD). 

4.  Unit-Test  Plan  -  A  description  of 
the  testing  approach  to  be  followed  in  debugging 
the  CPM  in  question.  The  unit-test  plan  includes 
the  overall  approach  to  module  testing  and  a 
description  of  each  test  case  required  to 
demonstrate  the  proper  operation  of  each 
function  performed  by  the  module.  All  test  case 
results  are  retained  in  the  UDF  for  the  CPM  in 
question. 

5.  Problem  Reports  -  Software 
problem  reports,  which  document  problems  sur¬ 
faced  during  the  integration  phase,  are  retained 
in  the  corresponding  CPM  unit  development 
folder.  These  problem  reports  and  their  re¬ 
solution  provide  a  chronicle  of  the  revisions 
undergone  by  the  CPM  in  question. 

A  software  problem  report  may  also  be 
prepared  to  document  an  enhancement  or  desired 
modification. 

Top-Down  Design.  The  MET  software 
design  was  developed  hierarchically  from  the  top 
down  by  the  development  of  a  software  work 
breakdown  structure  of  five  levels.  These  levels 
satisfy  the  objectives  of  groining  related 
functions  and  identifying  units  of  code  down  to 
the  lowest  level  controlled  and  beyond,  if 
necessary.  Figure  3  illustrates  the  structure  of 
MET  and  of  one  of  the  MET  software  subsystems 
to  the  lowest  level. 

1.  Computer  Program  (CP)  -  The  CP 
level  comprises  the  entire  deliverable  software 
product. 

2.  Computer  Program  Configuration 
Item  (CPCI)  -  The  CPCI  is  a  major  functional 
part  of  the  computer  program  and  may  be  of 
such  a  size  or  function  as  to  be  a  separately 
deliverable  and/or  contracted  item. 

3.  Computer  Program  Component 
(CPC)  -  The  CPC  is  a  subordinate  functional  part 


of  the  computer  program  which  is  usually 
grouped  with  other  CPCs  to  form  a  CPCI. 

4.  Computer  Program  Module  (CPM)  - 
The  CPM  is  the  smallest  functional  unit 
identifed  at  the  system  software  level  and 
controlled  by  configuration  management  pro¬ 
cedures.  CPMs  are  grouped  with  other  CPMs  to 
form  a  CPC. 

5.  Computer  Program  Submodule 
(CPSM)  -  The  CPSM  identifies  subroutines 
which  are  included  within  the  CPM. 
Submodules  are  not  controlled  by  configuration 
control  procedures  because  any  change  to  a 
submodule  is  considered  a  change  to  the  parent 
CPM  which  is  controlled. 

Module  Characteristics.  The  computer 
program  module  (CPM)  definition  adopted  for 
the  MET  project  is  characterized  as  follows: 

1.  Must  be  less  than  200  higher  order 
language  statements  in  length. 

2.  Must  perform  a  well  defined 
function. 

3.  Must  lend  itself  to  thorough  testing. 

4.  Must  be  an  aggregate  of  software  to 
which  specific  system  requirements  can  be 
traced. 

5.  Must  permit  changes  without  dis¬ 
turbing  the  overall  software  structure. 

Higher  Order  Language.  FORTRAN-77 
was  selected  as  the  implementation  language 
for  MET.  The  selection  was  based  on  the  fact 
that  FORTRAN-77  includes  the  structured 
programming  language  constructs  of  IF,  THEN, 
ELSE,  DO  UNTIL,  DO  WHILE,  etc.,  which 
facilitate  the  generation  of  clear 
easy-to-follow  code. 

Software  Design  Phase 

The  MET  software  design  phase  included 
those  activities  which  resulted  in  the 
preparation  of  the  contract  deliverable 
documents,  described  below,  and  the  conduct  of 
both  the  Preliminary  and  Critical  Design 
Reviews  (PDR,  CDR),  see  Figure  4. 

Software  Documents.  Five  basic  software 
documents  were  prepared  which  define  the 
system  specifications,  design  and  imple¬ 
mentation  .  Each  is  described  briefly  below. 

I.  Program  Performance  Specification 
(PPS)  -  The  PPS  document  is  based  upon  an 
analysis  of  the  contract  specification  and  de¬ 
scribes  the  functions  to  be  performed  by  the 
system.  This  description  includes  the 
equations/algorithms  which  are  to  be  imple¬ 
mented  to  simulate  real-time  functions  and  the 
performance  characteristics/operational  ranges 
of  these  functions.  Data  Input/Output  diagrams 
are  included  to  illustrate  ttw  flow  of  inputs  to 
and  outputs  from  the  various  system  function 
modules. 


446 


2.  Interface  Design  Specification  (IDS) 

-  The  IDS  document  is  based  upon  the  PPS,  the 
contract  specification  and  description  of  the 
system  hardware.  The  IDS  describes  in  detail 
the  flow  and  content  of  signals  and  data 
messages  passing  between  the  digital  compter 
processors  which  comprise  the  system  being 
implemented. 

3.  Program  Design  Specification  (PDS) 

-  The  PDS  document  is  based  upon  the  IDS, 
which  defines  the  data  interfaces  between  the 
digital  processors  in  the  system,  and  the  PPS, 
which  specifies  all  of  the  functions  to  be  per¬ 
formed  by  the  system.  The  PDS  provides  a 
description  of  how  the  system  being  designed 
will  satisfy  the  performance  requirements 
specified  in  the  PPS.  Included  is  a  definition  of 
the  software  architecture,  the  identification  and 
description  of  each  computer  program  module  to 
be  implemented,  the  inputs  and  outputs  required 
and  produced  by  each  CPM,  and  diagrams  which 
illustrate  the  flow  of  control  and  data  between 
the  CPMs  of  a  subsystem. 

4.  Program  Description  Document 
(PDD)  -  Preparation  of  the  POD  is  based  upon 
the  three  predecessor  documents  described 
above  and  includes  a  detailed  definition  of  the 
implementation  characteristics  of  each  CPM  in 
the  system.  The  PDD  constitutes  a  "code- to" 
level  document  and  includes  a  logic  flowchart  of 
each  CPM  and  CPSM  in  the  system,  accompanied 
by  a  narrative  keyed  to  the  flowchart,  which 
describes  the  processing  indicated.  A  summary 
of  the  data  base  items  used/produced  by  each 
CPM  and  CPSM  is  included  in  tabular  form  to 
support  the  implementation  description. 


fc-  -  I 


5.  Data  Base  Design  Document  (DBDD) 
-  The  DBDD  is  developed  in  parallel  with  the 
PDD  and  includes  a  detailed  definition  and 
description  of  each  data  item  stored  in  the 
system  data  base  and  accessed  by  the  real-time 
system  software.  Included  are  the  definitions 
of  varieties,  constants,  tables,  indexes  and 
flags.  The  combination  of  the  PDD  and  DBDD 
documents  provide  the  minimum  required  in¬ 
formation  needed  to  implement  the  system 
software. 


Design  Reviews.  Two  major  design 
reviews  were  conducted  on  the  MET  project. 
Each  is  briefly  discussed  below. 


1 .  Preliminary  Design  Review  (PDR)  - 
The  PPS  is  the  basic  document  subject  to 
review  during  the  PDR  and,  when  approved, 
constitutes  the  allocated  software  baseline. 
The  PPS,  associated  plans,  and  any  technical 
and/or  contractual  issues  are  reviewed  with  the 
customer  during  the  PDR  to  assess  the 
adequacy  of  the  system  performance  specifi¬ 
cation  and  software  allocation,  and  to  obtain  a 
mutual  commitment  to  proceed  to  the  detail 
design  phase. 

2.  Critical  Design  Reveiw  (CDR)  -  The 
CDR  is  conducted  following  the  preparation  of 
the  PDS  and  DBDD  which  constitute  the  soft¬ 
ware  product  baseline  at  the  CPM  level.  The 
CDR  is  normally  conducted  as  a  series  of 
formal  briefings  during  which  the  software 
design  is  related  to  the  specification 
requirements  and  described  in  sufficient  detail 
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FIGURE  4.  SOFTWARE  DEVELOPMENT  SEQUENCE 


♦o  demonstrate  that  all  system  requirements 
(processing,  timing,  simulation)  are  being 
satisfied  by  the  software  design.  The  CDR  is 
performed  to  assess  the  adequacy  of  the  soft¬ 
ware  product  design  and  to  obtain  an  agreement 
among  those  participating  to  proceed  to  the 
software  implementation  phase. 

Lessons  Learned.  A  number  of  problems 
were  encountered  and  resolved  during  the  MET 
design  phase  which  in  fact  have  provided  a 
valuable  learning  experience  which  hopefully  will 
be  applied  to  similar  software  developments  in 
the  future. 

1.  Standard  Development  Practices  - 
Assembly  of  the  MET  software  development 
team,  as  is  typical  with  many  large  software 
development  efforts,  was  an  event  which  re¬ 
sulted  in  the  joining  together  of  a  group  of 
software  engineers  with  varied  experiences  and 
backgrounds,  many  of  which  had  never  worked 
together.  Each  member  of  the  team  enriched 
the  group  with  unique  experiences;  however, 
each  such  contribution  was  accompanied  by 
personal  operational  predjudices  which  tended  to 
insure  that  a  different  design  and 
implementation  approach  would  be  followed  in 
developing  each  module  of  code. 

The  fusion  of  individual  viewpoints  and  the 
development  of  a  unified  approach  to  software 
development  was  accomplished  by  the 
preparation  of  a  set  of  software  development 
standards  that  were  adopted  and  followed  by  all 
to  both  design  and  implement  the  MET  software. 
These  standards  provided  a  basic,  if  not  always 
consistent,  set  of  rules  to  live  by.  All  members 
of  the  team  were  encouraged  to  recommend 
refinements  to  these  standards  as  the  project 
proceeded  from  one  phase  to  the  next.  Among 
the  major  development  standards  adopted  are 
the  following: 

o  Unit  Development  Folder  for  each  CPM 
developed 

o  Data  and  Program  Naming  Conventions 
o  Flowcharting  Standards 
o  Source  Code  Preface 
o  Code  Indentation  and  Labeling 
o  Design  Reviews 
o  Resource  Estimation 

Although  the  development  standards 
adopted  did  not  enjoy  universal  acceptance 
among  all  team  members  they  did  establish  a 
level  of  consistent  guidance.  Whether  accepted 
with  enthusiasm  or  resignation,  the  standards 
placed  everyone  in  the  same  circumstances. 
Despite  the  effort  to  standarize  there  were  a 
number  of  heroic  examples  of  deviation  from  the 
accepted  rules;  however,  I  believe  that  these 
were  minimized  by  the  existence  of  the 
standards. 

2.  Internal  Design  Reviews  -  Included 
in  the  development  standards  is  a  requirement 
for  the  performance  of  several  types  of  product 
reviews,  i.e.,  Design  Review,  Unit-Test  Plan 
Review,  Code  Review  and  Unit-Test  Review. 
The  reviews  are  conducted  at  various  stages  of 
development  and  are  briefly  described  below. 


a)  Design  Review:  Conducted 

during  the  preparation  of  the  preliminary 
Program  Description  Document  and  consisting 
of  a  walkthrough,  by  the  responsible  software 
engineer,  of  the  CPM  flowchart  with  an  atten¬ 
dant  discussion  of  the  CPM  inputs,  processing, 
outputs  and  interactions  with  other  CPMs. 

b)  Unit- Test  Plan  Review:  Usually 
performed  at  or  near  the  end  of  code 
generation  and  consisting  of  an  examination  of 
the  CPM  unit-test  plan  for  adequacy  of  CPM 
demonstration.  All  test  cases  are  examined 
along  with  inputs  and  expected  processing  out¬ 
puts. 

c)  Code  Review:  Conducted  follow¬ 
ing  code  generation  and  consisting  of  a  review 
of  the  code  logic  to  insure  compliance  with  the 
logic  contained  on  the  CPM  flowchart  and  a 
review  of  the  code  comments  to  insure  a  direct 
correlation  with  the  PDD  flowchart  narrative. 


At  the  time  of  review,  all  flowchart  corrections 
must  be  red- lined  as  well  as  the  corresponding 
PDD  narratives  describing  the  flowchart. 

d)  Unit-Test  Review:  Performed  at 
conclusion  of  CPM  unit-testing  (debugging)  and 
consisting  of  a  careful  review  of  the  CPM 
unit-test  plan,  all  test  cases,  the  inputs  to 
those  test  cases  and  a  comparison  of  the 
computed  outputs  with  the  anticipated  outputs 
as  documented  in  the  test  case  descriptions.  At 
the  time  of  Unit-Test  Reveiw  all  software 
documents  (PPS,  PDS,  PDD  and  DBDD)  must 
have  been  corrected  (red-lined)  to  reflect  the 
unit-tested  program. 

The  completion  of  unit-test  plan,  code, 
code  review,  unit-testing  and  unit-test  reviews 
were  events  which  were  monitored  formally  on 
a  weekly  basis.  Considerable  management 
attention  was  given  to  these  reviews  to  insure 
that  each  CPM  was  thoroughly  tested  prior  to 
being  released  to  the  integration  team  for 
integration  into  the  system  and  to  insure  that 
the  "as  coded"  CPM  was  properly  documented 
hence  minimizing  the  need  for  a  crash 
documentation  update  cycle  at  project  end. 

3.  Document  Preparation  -  The 
preparation  of  software  documents  in 
accordance  with  the  SECNAVINST  3560.1  and 
Mil  Std  1644  documentation  standards  was  a 
more  formidable  task  than  had  been  projected. 
The  serial  manner  in  which  the  documents  were 
required  by  the  contract  schedule  guaranteed 
that  differences  and  inconsistencies  would  exist 
in  each  succeeding  document.  On  the  average 
the  time  scheduled  between  the  PPS,  PDS,  PDD 
and  DBDD  documents  was  approximately  three 
months. 

During  each  succeeding  document  pre¬ 
paration  period  a  better  understanding  of 
system  operation  and  design  was  obtained.  As 
this  more  complete  system  understanding  found 
its  way  into  the  document  being  generated, 
differences  between  the  new  document  and  the 
previous  documents)  were  created. 
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This  condition  was  the  source  of  numerous 
difficulties  with  customer  personnel  who  were 
reviewing  the  documents  for  consistency  and 
traceability.  The  end  result  of  this  situation  was 
that  a  major  document  update  was  performed  to 
bring  all  the  software  documents  to  the  same 
level  of  design,  an  activity  which  seriously 
impacted  the  start  of  implementation. 

The  problem  of  discrepancies  between 
documents  generated  serially  on  a  tight  schedule 
would  have  been  practically  eliminated  if  a 
period  of  time  had  been  incorporated  in  the 
schedule  at  the  time  of  each  document's 
completion  during  which  the  previous 
document(s)  could  have  been  updated  to  the 
design  level  of  the  most  recent  document. 


Software  Implementation  Phase 

The  MET  software  implementation  phase 
included  those  activities  which  resulted  in  the 
generation  of  code,  the  procedures  for  testing 
that  code  and  the  conduct  of  reviews  to  insure  a 
software  product  that  performed  the  functions 
for  which  it  was  designed,  see  Figure  4.  The 
implementation  sequence  followed  in  the  MET 
system  was:  development  of  a  unit-test  plan; 
code  generation;  code  review;  unit- test;  and 
unit- test  review. 

Unit-Test  Plan.  A  unit-test  plan  was 
prepared  for  each  Cf’M  implemented  in  the  MET 
system.  The  unit-test  plan  described  the 
approach  to  be  followed  in  testing  o  CPM  and 
included  a  description  of  each  test-case  required 
to  demonstrate  the  proper  operation  of  each 
function  performed  by  a  CPM.  Each  test  case 
description  included  the  inputs  used  in  the  test 
and  the  projected  outputs  to  be  computed. 
Normally  the  unit-test  plan  was  developed  in 
parallel  with  the  generation  of  code  for  a  CPM. 

Code  Generation  and  Review.  MET  code 
was  written  using  the  FORTRAN-77  language 
and  the  coding  standards  adopted  at  the  start  of 
the  project.  Only  certain  time-critical  CPMs 
were  coded  in  assembly  language.  Review  of  the 
code  required  a  "clean  compile",  not  a  debugged 
or  unit-tested  program.  Code  Reveiw  was 
performed  by  the  chief  programmer  under  whose 
direction  the  CPM  was  being  developed.  Review 
consisted  of  an  examination  of  the  code  to 
insure: 

a)  Compliance  with  the  coding 

standards 

b)  Correlation  between  the  code 
logic  and  the  CPM  flowchart  logic 

c)  Correlation  between  the  code 
comments  and  the  flowchart  narrative  (POD) 

d)  Corrected  (red-lined)  CPM 
flowchart  and  narrative  to  reflect  the  "as  coded" 
state  of  the  CPM. 

Unit-Test  and  Review.  The  Unit- Test 
activity  consisted  of  debugging  a  CPM  in 
accordance  with  the  unit-test  plan  and  its  test 
cases.  The  output  results  of  each  test  case  were 


saved  for  the  unit-test  review.  CPM 
unit-testing  is  normally  accomplished  in  a 
static  environment,  i.e.,  non  real-time,  one 
CPM  at  a  time.  A  number  of  test  tools  were 
developed  for  use  on  the  MET  project  which 
facilitated  the  execution  and  test  of  a  CPM(s) 
in  both  the  static  and  dynamic  environmments. 


When  a  programmer  felt  that  a  CPM  was 
completely  operational,  a  unit-test  review  was 
held.  This  review  was  performed  by  a  group  of 
concerned  software  engineers  which  included 
all  of  the  chief  programmers.  In  attendance 
also  were  other  programmers  whose  CPMs 
might  interact  with  the  CPM  being  reviewed. 

The  responsible  programmer  presented 
the  CPM  design,  the  unit-test  plan,  a  discussion 
of  the  test  cases  run  and  provided  all  of  the 
test  case  outputs  for  examination  by  the  review 
team.  Any  problems  surfaced  in  the  course  of 
the  review  were  documented  by  the  attending 
software  quality  management  person  and  a 
schedule  established  for  the  resolution  of  those 
problems. 

Following  successful  conclusion  of  a 
unit-test  review,  a  CPM  was  considered  ready 
for  advance  to  the  integration  phase.  From  this 
point  on  the  CPM  source  code  was  controlled  by 
the  software  librarian. 


Lessons  Learned.  The  software  imple- 
mentafion  phase  was  highlighted  not  only  by  the 
traditional  difficulties  of  monitoring  status  but 
also  by  a  few  pleasant  surprises  as  described 
below. 

1.  Code  Generation  -  The  MET 
implementation  phase  was  preceded  by  a  design 
phase  which  was  characterized  by  the 
preparation  of  software  documentation  (PPS, 
PDS,  PDD,  DBDD).  The  generation  of  these 
documents,  which  culminated  in  a  preliminary 
"code-to  level"  PDD,  produced  a  software 
design  that  had  been  thought  through.  As  a 
result,  the  code  generation  was  accomplished 
much  more  rapidly  than  anticipated. 

2.  Unit-Test  Plan  -  It  was  determined 
that  development  of  a  CPM  unit-test  plan  was 
accomplished  most  effectively  when  generated 
in  parallel  with  or  following  actual  code 
generation.  When  developed  prior  to  code 
generation,  more  time  was  required  to  produce 
the  unit-test  plan  and  invariably  it  had  to  be 
modified  slightly  after  CPM  coding. 

3.  Unit-Testing  -  The  use  of  the 
Program  Module  Test  (PMT)  facility,  for 
unit-testing  of  indiviAxsl  modules,  was  felt  to 
be  quite  effective.  PMT  is  a  test  tool  which 
accepts  testing  commands,  in  card  format, 
which  result  in  initial  values  being  set,  a  CPM 
being  brought  into  execution  a  specified  number 
of  times  and  selected  register  and/or  memory 
locations  being  printed  as  the  CPM  is  executed. 
An  additional  value  of  PMT  was  that  the  test 
card  deck  could  be  used  over  and  over  as 
changes  were  made  to  a  CPMH  code,  hence, 
keeping  the  testing  base  constant. 
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Other  software  test  tools  were  developed 
which  permitted  the  testing  of  several  CPMs  in 
parallel  in  a  dynamic  or  real-time  environment. 

4.  Implementation  Status  -  A  number 
of  methods  were  used  to  assess  implementation 
progress.  Several  are  discussed  below. 

a)  PERT  Type  Charts  -  This  type  of 
chart  is  o  widely  accepted  management  visual; 
however,  it  can  be  quite  difficult  to  interpret. 
The  MET  software  PERT  type  charts  were 
color-coded  with  black,  meaning  a  task  or 
activity  accomplished,  red,  meaning  not  done 
and  behind  schedule  and  green,  for  done  and 
ahead  of  schedule.  On  more  than  one  occasion 
task  performance  sequence  was  changed  because 
certain  routines  became  more  critical  than 
others,  hence,  were  started  out  of  the  planned 
sequence.  Whenever  this  situation  occurs  the 
software  manager  is  faced  with  having  to  explain 
that  the  presence  on  the  chart  of  incomplete 
tasks  (red)  is  unimportant  to  the  overall  schedule 
or  that  the  red  is  balanced  by  the  presence  of 
tasks  completed  early  (green).  The  effort 
required  to  explain  this  type  chart,  and  put  the 
program  manager  at  ease,  can  often  be  greater 
than  that  required  to  monage  the  development 
itself. 

b)  Progress  Graphs  -  A  number  of 
graphs  were  prepared  and  updated  on  a  weekly 
basis  which  plotted  actual  accomplishment 
against  the  plan.  For  example  a  graph  was 
generated  for  the  completion  of  each  software 
development  event,  i.e,  Unit-Test  Plan,  Coding, 
Code  Review,  Unit-Test  and  Unit-Test  Review. 
Plotting  of  the  actual  event  completions  against 
planned  completions  showed  very  clearly  when 
and  if  the  implementation  activity  was  ahead  of 
schedule  or  behind.  These  graphs  were  used 
quite  effectively  to  provide  progress  status  to 
management  as  well  as  the  developers 
themselves. 

c)  Status  Meetings  -  Maintaining 
individual  motivation  on  a  software  development 
project  of  long  duration  which  is  frequently 
beset  by  deadlines  is  a  diplomatic  and 
management  challenge  of  some  proportion.  On 
MET  it  has  been  accomplished  by  holding  weekly 
software  status  meetings  with  all  of  the 
members  of  a  team  in  attendance.  Status  is 
obtained  by  reviewing  each  item  due  that  week 
and  determining  if  the  item  is  complete,  if  not, 
why  not,  and  when  necessary  establishing  a  new 
promise  date  for  that  item. 


Quite  obviously  this  type  of  inquiry  places 
each  individual  in  the  limelight  which,  on 
occasion,  is  uncomfortable.  Initially  the  status 
meetings  were  not  very  popular  and  were  the 
source  of  considerable  grumbling  associated  with 
unrealistic  schedule  dates,  inoperative  hardware, 
unavailable  information  which  impacted  task 
completion,  etc.  However,  after  the  first  six  or 
seven  meetings  a  marked  change  in  attitude 
became  apparent.  Individual  engineers  who  had 
been  given  the  opportunity  to  modify  schedule 
dates  now  took  them  much  more  seriously. 
Meeting  the  completion  dates  estimated  became 
a  matter  of  professional  pride. 


The  weekly  status  meetings  provided  a 
forum  where  management  could  applaud 
individual  and  team  performance  to  the 
schedule  and  where  problems,  which  impocted 
software  development,  could  be  aired  and 
resolved.  The  overall  effect  of  the  status 
meetings  on  team  motivation  was  quite 
positive. 

Software  Integration  Phase 

The  software  integration  phase  included 
those  activities  which  resulted  in  assembling  all 
of  th/'  software  components  into  subsystems, 
exe.  .ng  the  functions  of  each  subsystem 

independently,  isolating  and  resolving 
processing  anomalies,  exercising  all  subsystems 
in  parallel  and  producing  a  total  system  which 
operates  properly  with  the  hardware  base, 
performing  ail  of  the  functions  for  which  the 
system  was  designed.  The  procedures  followed 
to  control  software/system  integration  are 
briefly  described  below. 

Integration  Team.  Integration  of  the 
software  CPMs  into  an  operational  system  is 
an  activity  which  is  performed  by  an 
integration  team.  The  integration  team  is 
composed  of  two  or  three  of  the  most 

knowledgeable  people  from  each  subsystem 
area.  The  team  is  responsible  for  development 
of  an  integration  plan,  the  generation  of 
unit-test  plan  procedures,  describing  how  the 
various  system  functions  are  to  be  exercised, 
and  for  the  actual  execution  of  those 

procedures. 

It  must  be  noted  here  that  a  considerable 
effort  was  expended  during  the  implementation 
phase  to  insure  that  each  CPM  had  been 

extensively  unit-tested.  Each  CPM  under¬ 
went  a  thorough  unit-test  review  where 
computed  output  was  verified,  hence,  every 
attempt  was  made  to  insure  that  each  CPM  was 
an  operational  package  prior  to  the  integration 
of  that  CPM  into  the  system.  Hopefully  the 
precautions  taken  will  minimize  the  integration 
effort  required. 

Integration  will  be  carried  out  on  various 
levels  from  the  verification  of  single-thread 
system  functions,  with  or  without  program 
stubs,  to  mulitple  processes  operating  in 
parallel  in  real-time.  Unit-test  plans  are 
generated  for  each  level  of  system  testing. 

Software  Problem  Reports.  As  may  be 
expected,  software  problems  will  be  surfaced  in 
the  course  of  integrating  the  system.  These 
problems  will  be  documented  on  Software 
Problem  Report  forms  by  the  integration  team 
members  and  monitored  by  the  software 
Quality/Management  (Q/M)  person.  The 
problems  identified  during  integration  may  be 
assigned  to  the  original  programmer  for  re¬ 
solution,  if  still  available,  or  they  will  be 
resolved  by  the  integration  team  directly. 

As  resolutions  are  implemented  the  soft¬ 
ware  Q/M  person  will  be  involved  to  verify  that 
the  change  has  been  made  and  to  record  the 
need  for  documentation  modifications  as 
required. 
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Configuration  Control.  At  the  conclusion 
of  CPM  implementation  (successful  completion 
of  the  unit-test  review),  a  CPM  is  considered  to 
be  thoroughly  debugged  and  to  perform  in 
accordance  with  the  documented  design.  One  of 
the  requirements  of  the  unit-test  review  is  for 
the  responsible  programmer  to  turn  over  the 
CPM  source  ond  object  code  to  the  Software 
Librarian  who  obtains  a  revision  number  for  the 
CPM  from  the  software  Q/M  person,  and  records 
it  in  the  system  library  where  it  becomes 
available  to  the  integration  team  for 
incorporation  into  the  system. 

Control  of  CPM  source  and  object  code 
will  be  maintained  by  the  software  Q/M  person 
and  the  software  librarian  throughout  the 
integration  period.  Any  and  all  modifications  to 
be  incorporated  into  a  CPM  will  be  made  by  the 
software  librarian  to  the  CPM  source  which  is 
then  recompiled,  has  a  new  revision  number 
assigned,  and  is  re-recorded  in  the  system 
library  for  use  by  the  integration  team.  This 
control  procedure  eliminates  surprise  nocturnal 
fixes  made  by  programmers  to  CPMs  which  are 
actively  being  integrated. 

Lessons  Learned.  At  the  time  of  this 
writing,  integration  of  the  MET  software  system 
has  not  yet  been  initiated,  hence,  it  is  premature 
to  discuss  integration  advantages  which  may  be 
realized.  It  is  believed,  however,  that  the 
integration  should  go  more  smoothly  based  upon 
the  extensive  design  documentation  prepared  in 
advance  of  implementation,  and  of  the  thorough 
unit-testing  performed  and  verified  for  each 
CPM  prior  to  submission  for  integration. 

SUMMARY 

Management  control  and  technical  development 
of  the  MET  software  has  been  characterized  by 
the  development  and  adoption  of  meaningful 
software  development  practices  and  procedures, 
and  adherence  to  the  concept  that  software 
design  must  precede  implementation.  This 
activity  has  provided  valuable  learning 
experiences  in  a  lumber  of  areas,  some  of  which 
are  outlined  belov.  Inasmuch  as  the  project  has 
not  entered  into  the  integration  phase, 
comments  are  confined  to  the  design  and 
implementation  pfiases  of  MET  development. 

Design  Documentation 

The  military  standards  required  and 
utilized  on  the  MET  project  (SECNAVINST 
3560.1  and  Mil  STd  1644)  have  required  much 
more  time  and  manpower  to  satisfy  than  was 
originally  planned. 

Adherence  to  these  standards,  however, 
does  impose  a  discipline,  namely,  that  one  must 
have  considered  the  design  thoroughly  if  it  is  to 
be  described  in  the  detail  required  by  these 
standards.  This  is  an  investment  which  has 
begun  to  pay  off  in  the  software  implementation 
phase  and  hopefully  will  continue  to  pay  off 
during  integration. 

An  attempt  should  be  made  at  project 
initiation  to  incorporate  sufficient  schedule  time 


between  each  document  generation  to  permit 
updating  previous  documents  to  the  same  level 
of  design.  If  this  is  not  possible  it  should  be 
understood  and  accepted  that  differences  and 
inconsistencies  will  exist  in  the  set  of 
documents  produced. 


Software  Standard  Practices 

It  is  absolutely  necessary  that  a  fairly 
rigid  set  of  software  development  practices  be 
available  if  there  is  to  be  any  hope  of  molding  a 
professional  team  that  will  be  moving  in  the 
same  direction  toward  a  common  system  goal. 

Although  somewhat  restrictive  of  in¬ 
dividual  professional  styles,  the  standards 
enforce  a  consistency  which  on  a  sizeable 
project  is  often  more  valuable  than  individual¬ 
ism.  The  standards  themselves  must  be  subject 
to  continual  refinement  by  recommendation 
from  any  member  of  the  software  staff.  People 
identify  with  a  set  of  standards  that  they  can 
help  to  formulate. 


Software  Implementation 

As  a  result  of  the  requirement  for  design 
documentation  prior  to  implementation  and  the 
establishment  of  standards,  the  generation  of 
code  has  occurred  much  more  rapidly  than 
anticipated. 

The  design  errors  surfaced  in  the  course 
of  code  generation  have  also  tended  to  be  more 
minor  than  in  previous  projects. 

Unit-testing  (debugging)  of  individual 
CPMs  appears  to  be  progressing  more  quickly 
than  anticipated  for  the  same  reasons. 

Development  Team  Motivation 

The  experience  gained  on  the  MET  project 
indicates  that  the  most  effective  way  to 
maintain  a  high  level  of  team  motivation  is  to 
monitor  performance  regularly.  Weekly  status 
meetings  were  the  vehicle  utilized  on  MET. 

Performance  monitoring  using  this 
methodology  provides  a  technique  for  surfacing 
problems  which,  upon  verbalization,  may  be 
shared  by  a  number  of  team  members. 
Obviously  if  a  problem  is  not  surfaced  it  cannot 
be  solved.  Status  meetings  were  extremely 
helpful  in  this  regard. 

Each  item  reviewed  in  the  status  meeting 
was  subject  to  schedule  modification,  if  the 
conditions  so  required,  a  situation  which 
enabled  a  team  member  to  reflect  a  sense  of 
realism  and  personal  commitment  in  the 
development  plan. 

The  status  meetings,  which  were  held 
weekly  during  the  implementation  phase, 
provided  management  with  on  opportunity  to 
display  an  interest  in  what  team  members  were 
doing  and  to  recognize  individual  and  team 
achievements. 
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The  result  of  the  status  rrsetings  was 
plotted  on  five  graphs  depicting  progress  versus 
plan  for  each  of  the  primary  implementation 
activities  (Coding,  Unit-Test  Plan,  Code  Review, 
Unit-Test,  Unit- Test  Review).  These  graphs 
were  prominently  displayed  for  all  to  assess  their 
progress. 

Another  factor  which  has  played  no  small 
part  in  maintaining  positive  team  motivation  has 
been  an  involved,  knowledgeable  customer.  The 
MET  trainer  is  being  developed  by  the  CUBIC 
Corporation  for  the  Naval  Training  Equipment 
Center  (NTEC)  Orlando,  Florida.  NTEC 
personnel  have  been  personally  involved  with  the 
MET  design  and  software  developments  and  have 
spent  many  hours  reviewing  both  the  design  and 
implementation  documents  at  the  CPM  and 
CPSM  level.  Their  participation  in  the  MET 
development  has  contributed  to  a  sound 
technical  relationship,  with  each  of  the  chief 
programmer  teams,  and  has  resulted  in  a 
continued  and  productive  dialog  between 
developer  and  customer. 
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ABSTRACT 

This  paper  will  discuss  the  application  of  software  quality  assurance  tech¬ 
niques  to  trainer  software  development,  taking  into  consideration  military  standards 
and  specifications  and  the  unique  characteristics  of  trainer  development  programs. 
8ecause  military  customers  are  paying  greater  attention  to  software  development  and 
software  documentation,  software  quality  assurance  has  become  an  increasingly  impor¬ 
tant  management  objective.  Although  there  are  no  quick  recipes  for  adapting  soft¬ 
ware  quality  assurance  techniques  and  standards  to  trainer  development,  this  objec¬ 
tive  can  be  met  by  analyzing  the  software  specifications  and  standards  along  with 
the  software  quality  assurance  specifications  and  standards,  by  considering  the 
unique  aspects  of  trainer  development,  and  by  considering  the  aspects  of  applying 
and  adapting  software  standards  to  trainer  development.  First,  the  various  speci¬ 
fications  and  standards  that  apply  to  software  development  must  be  analyzed  with 
particular  attention  paid  to  their  Interrelationships  and  to  their  relationships 
with  deliverable  data  items.  Then,  this  conceptual  framework  must  be  related  to  the 
software  quality  assurance  standards  and  specifications.  Differences  and  similari¬ 
ties  between  the  standards  and  specifications  written  by  different  military  custo¬ 
mers  will  also  be  considered.  Given  this  overall  picture  of  the  requirements  for 
software  development  and  quality  assurance,  the  unique  aspects  of  trainer  develop¬ 
ment  may  be  considered.  Among  these  are  shortened  schedules,  abbreviated  data  re¬ 
quirements  and  the  application  of  weapon  system  standards  to  trainer  development. 
Once  the  various  requirements  and  the  peculiar  constraints  of  trainer  development 
have  been  analyzed,  the  next  step  is  to  consider  the  meaningful  application  of 
software  standards  and  quality  assurance  techniques  to  trainers.  Among  these  con¬ 
siderations  are  cost  effectiveness,  who  should  accomplish  the  various  quality  assur¬ 
ance  tasks,  applicability  of  internal  standards,  whether  tasks  are  best  handled  on  a 
company  or  program  basis,  tailoring  quality  assurance  functions  to  program  needs, 
and  the  problem  of  assuring  quality  of  software  when  there  are  no  specific  software 
data  item  requirements.  In  conclusion,  this  paper  will  present  an  approach  to 
developing  a  software  quality  assurance  program  for  trainer  system  development. 


Software  management  and  control  presents  the 
same  challenge  to  the  trainer  industry  that  it 
does  to  major  computer  and  weapons  system  manufac¬ 
turers.  Although  the  overall  development  cost  for 
a  training  system  may  be  only  a  fraction  of  the 
cost  for  the  system  it  simulates,  the  percentage 
of  development  cost  allocated  to  software  may  be 
greater  in  the  training  system  because  the  trainer 
industry  has  become  increasingly  software  inten¬ 
sive  and  there  is  every  indication  that  this  trend 
will  continue.  Given  the  fact  that  the  cost  of 
software  development  comprises  a  larger  portion  of 
the  cost  of  system  development  each  year,  control¬ 
ling  software  development  has  become  a  primary 
management  concern. 

Methods  and  procedures  for  control  of  cost 
and  quality  of  hardware  development  have  been 
applied  In  Industry  for  so  long  that  they  have 
become  almost  second  nature  and  we  no  longer  ask 
whether  or  not  procedures  such  as  an  engineering 
drawing  system  or  a  hardware  change  control  system 
should  be  instituted.  Now  that  control  of  soft¬ 
ware  development  has  become  more  of  a  concern,  we 
cannot  treat  the  software  portion  of  system  deve¬ 
lopment  as  a  black  box  design  for  which  we  dedi¬ 


cate  money  and  manpower  at  the  beginning  of  a 
program,  only  to  have  it  slip  from  view  until  test 
and  integration  when  it  either  works  or  does  not 
work.  But  it  is  obvious  that  the  same  controls 
that  have  been  applied  to  hardware  development 
cannot  be  applied  directly  to  software  development 
and  also  that  controls  applied  to  major  system 
software  development  might  not  be  feasible  or  cost 
effective  when  applied  to  trainer  system  software 
development. 

Control  of  software  development  is  generally 
divided  into  management  functions  and  quality 
assurance  functions.  Software  management  Includes 
functions  Intended  to  specify  and  control  the 
design,  and  software  QA  Includes  functions  in¬ 
tended  to  monitor  the  design  process  according  to 
the  management  objectives.  A  number  of  software 
management  and  QA  techniques  and  procedures  have 
been  developed  which  can  provide  models  for  train¬ 
er  software  management  and  QA,  and  provide  techni¬ 
ques  that  may  be  adapted  to  fit  the  needs  of  a 
trainer  development  program.  Since  the  government 
is  the  major  buyer  and  user  of  software,  it  stands 
to  reason  that  the  control  of  software  design 
would  be  a  major  government  concern.  This  concern 
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is  reflected  in  military  standards  and  specifica¬ 
tions,  which  include  a  rather  complete  blueprint 
for  software  development.  It  should  be  noted  that 
the  software  management  and  QA  techniques  employed 
by  many  individual  companies  are  either  derived 
from  or  reflected  in  these  standards.  This  is  not 
to  say  that  the  military  standards  are  the  last 
word  in  software  development  or  that  they  are 
unilaterally  complete  or  effective,  but  they  are  a 
useful  source  for  companies  desiring  to  institute 
software  management  and  control  techniques  and  an 
effective  software  QA  effort. 

The  military  standards  that  describe  the 
software  development  effort  are  MIL-STD-483, 
MIL-STD-490  and  MIL-STD-1521,  and  documents  which 
are  devoted  primarily  to  software  QA  are  MIL-S- 
52779,  MIL-STD-1679  and  MIL-STD-1644.  Briefly, 
MIL-STD-483  can  be  seen  as  the  grandfather  of  this 
group  of  documents  because  it  details  the  content 
requirements  for  software  documentation  along  with 
MIL-STD-490.  MIL-STD-1521  specifies  the  documen¬ 
tation  required  at  the  completion  of  the  various 
phases  of  software  development.  MIL-S-52779  calls 
for  the  implementation  of  a  software  QA  program 
and  could  be  described  as  a  parallel  document  of 
MIL-Q-9858,  which  specifies  quality  control  re¬ 
quirements  for  hardware  development.  MIL-STD- 
1679,  a  Navy  document,  includes  more  detailed 
software  quality  program  requirements  and  refer¬ 
ences  a  set  of  software  documentation  which  is 
similar  to  that  described  by  MIL-STD-483.  Fin¬ 
ally,  MIL-STD-1644  is  basically  a  MIL-STD-1679 
approach  applied  to  trainers  with  very  little 
tailoring. 

Two  basic  concepts  are  common  to  all  of  these 
documents  and  these  two  concepts  can  be  applied  to 
form  the  basis  for  a  trainer  software  management 
and  QA  effort.  The  first  of  these  is  the  concept 
of  development  phases.  That  is,  the  software 
development  process  is  divided  into  successive 
phases  marked  by  milestones,  at  which  the  design 
up  to  that  point  is  analyzed  and  validated.  The 
second  concept  comes  into  play  In  the  form  of 
documentation.  Each  milestone  is  represented  by  a 
documentation  set  which  may  consist  of  a  specifi¬ 
cation  or  flow  charts  or  printouts,  depending  upon 
the  particular  phase  that  is  represented.  This 
documentation  must  be  seen  as  a  management  tool 
which  provides  visibility  of  the  design  and  which 
provides  sufficient  detail  to  validate  the  design. 
The  documentation  also  specifies  what  will  be 
accomplished  in  the  next  phase  so  that  at  the 
completion  of  each  phase,  the  documentation  may  be 
validated  by  comparing  it  with  the  documentation 
generated  in  the  previous  phase.  This  provides 
for  traceability  of  the  design  to  the  original 
system  requirements  throughout  all  phases  of 
development.  A  simplified  schedule  for  a  typical 
software  development  effort  is  shown  in  Figure  1. 

For  the  purposes  of  this  discussion,  the  software 
development  process  will  be  divided  Into  four 
major  phases:  Analysis,  Design,  Coding,  and  Test 
and  Integration.  Other  divisions  are  possible, 
provided  each  phase  leads  to  a  milestone  which  can 
be  seen  as  a  design  baseline  represented  by  a 
specific  set  of  documentation  that  describes  the 
design  up  to  that  point.  The  terms  used  to  de¬ 
scribe  the  documentation  will  be  taken  from  MIL- 
STO-483.  recognizing  that  the  MIL-STO-1679  differs 
primarily  in  form,  rather  than  intent. 


If  we  envision  a  software  development  effort 
beginning  with  a  decision  to  allocate  funds  to  a 
certain  project  or  beginning  with  a  contract 
award,  the  first  task  is  to  analyze  the  system 
software  requirements  for  testability,  functional 
grouping,  and  implementation.  Testability  is  a 
determination  of  whether  or  not  the  system  re¬ 
quirements  can  be  implemented  and  tested,  because 
once  the  design  is  complete  we  must  be  able  to 
prove  that  all  system  requirements  have  been  met. 
Also  during  the  Analysis  phase,  the  system  require¬ 
ments  might  be  divided  into  two  or  more  functional 
areas,  often  called  Computer  Program  Configuration 
Items,  if  it  appears  smaller  units  of  design  may 
be  easier  to  control  than  one  larger  system.  The 
major  activity  of  this  phase  is  the  translation  of 
system  requirements  into  software  requirements. 
That  is,  we  analyze  the  overall  requirements  and 
come  up  with  a  set  of  Instructions  that  can  be 
understood  and  followed  by  a  software  designer. 
These  instructions  are  contained  in  a  Computer 
Design  Specification,  or  a  similar  document,  for 
each  Computer  Program  Configuration  Item.  This 
document  will  include  such  information  as  block 
diagrams,  general  flow  diagrams,  interface  defi¬ 
nitions,  and  software  system  requirements.  Often 
the  milestone  which  closes  the  Analysis  phase  is 
the  Preliminary  Design  Review,  during  which  the 
Computer  Program  Design  Specifications  are  com¬ 
pared  to  the  overall  system  requirements  and  all 
necessary  changes  are  incorporated. 

During  the  Design  phase,  the  software  de¬ 
signers  follow  the  requirements  of  the  design 
specification  and  draft  the  Computer  Program 
Product  Specification.  Among  the  activities  in 
this  phase  are  the  preparation  of  detailed  flow 
charts,  developing  a  top-down  module  structure, 
and  allocating  the  available  time  and  memory. 
This  phase  culminates  in  a  Critical  Design  Review 
during  which  the  Computer  Program  Product  Speci¬ 
fications  are  compared  to  the  design  specifica¬ 
tions  and  validated,  once  all  changes  are  made. 
The  Computer  Program  Product  Specification  is  a 
draft  document  at  this  point  because  it  will  be 
updated  to  include  listings  once  the  Coding  and 
Test  phases  are  complete. 

Following  the  Critical  Design  Review,  the 
product  specifications  may  be  turned  over  to  the 
programmers  for  coding.  Completion  of  this  effort 
results  in  a  milestone  review  which  may  take  the 
form  of  a  design  review  or  a  similar  activity, 
such  as  a  code  walk-through.  The  purpose  of  this 
milestone  is  to  check  the  code  against  the  ap¬ 
proved  flow  charts  contained  in  the  product  speci¬ 
fication  before  the  code  is  entered  into  the 
computer  load.  This  may  be  done  on  a  module  by 
module  basis  or  in  larger  units.  The  documents 
that  describe  this  milestone  are  the  product 
specifications,  which  are  updated  to  include  the 
approved  code. 

A  parallel  branch  of  the  development  effort 
which  has  not  been  mentioned  so  far  is  the  deve¬ 
lopment  of  test  plans,  test  procedures  and  test 
software.  During  the  Analysis  phase  a  plan  to 
test  the  completed  software  should  be  established 
and  the  necessary  procedures  written.  Any  test 
and  analysis  software  that  will  be  required  to 
verify  system  requirements  should  be  designed  and 
documented  In  the  same  way  that  the  rest  of  the 
system  software  is  designed.  That  is,  test  soft- 
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ware  should  not  be  a  last  minute  consideration. 
It  may  be  helpful  to  allocate  a  Computer  Program 
Configuration  Item  to  the  test  software  during  the 
Analysis  phase  to  ensure  that  it  will  be  ready  and 
operable  when  testing  begins.  Test  plans  and 
procedures  should  include  provisions  to  test  every 
requirement  of  the  system  specification. 

The  Test  and  Integration  phase  usually  begins 
with  some  form  of  module  testing.  Modules  may  be 
verified  against  the  design  and  product  specifi¬ 
cations  according  to  the  approved  test  plan  and 
procedures.  Verified  modules  are  compiled  into 
their  respective  Computer  Program  Components  and 
further  testing  is  conducted  to  verify  the  entire 
component  before  it  is  compiled  into  the  Computer 
Program  Configuration  Item.  The  complete  config¬ 
uration  item  may  then  be  validated  against  the 
design  and  product  specifications.  Once  we  have 
verified  the  software  load,  integration  with  the 
hardware  may  take  place,  and  this  process  culmin¬ 
ates  in  a  testing  procedure  which  validates  the 
entire  system  against  the  system  specification. 
Following  successful  completion  of  all  testing, 
the  updated  product  specifications  define  the 
production  baseline  for  the  computer  program 
system. 

So,  we  began  with  the  system  specification, 
gleaned  from  that  document  the  software  objec¬ 
tives,  defined  specific  software  design  tasks, 
divided  the  design  effort  into  phases  that  corres¬ 
pond  to  visible  baselines,  which  were  themselves 
defined  by  a  specific  set  of  documentation,  and 
then  compiled  the  software  packages  into  a  com¬ 
plete  system  which  we  verified  against  the  system 
specification. 

The  objective  of  the  software  QA  effort  is  to 
monitor  this  process  and  ensure  that  the  system 
requirements  will  be  met.  Again  using  the  mili¬ 
tary  specifications  and  standards  as  a  model,  the 
software  QA  program  tasks  will  include  documen¬ 
tation  reviews,  participation  in  design  reviews 
and  walk-throughs,  independent  audits  of  design 
procedures,  corrective  action  and  trend  analysis, 
and  configuration  control.  Included  in  the  con¬ 
figuration  control  tasks  are  monitoring  of  library 
controls,  review  of  test  reports,  and  change  con¬ 
trol  . 

In  general,  there  are  four  design  character¬ 
istics  that  the  software  QA  representatives  will 
hope  to  identify  during  the  QA  reviews  and  audits. 
The  first  of  these  characteristics  is  traceability 
of  design  requirements,  which  means  simply  that  at 
each  milestone  of  the  design  process,  we  should  be 
able  to  trace  each  element  in  the  software  docu¬ 
mentation  back  through  the  product  specification 
and  the  design  specification  to  the  system  require¬ 
ments.  If  the  documentation  .'.as  been  completed 
correctly  and  if  the  design  has  been  documented 
completely,  management  confidence  in  the  progress 
of  the  design  will  be  gained  from  the  reviews  and 
audits.  If  design  traceability  has  been  lost, 
this  problem  will  be  evident  in  reviews  and  audits 
and  there  will  be  less  confidence  that  the  design 
will  eventually  meet  all  system  requirements. 
Secondly,  the  various  QA  reviews  and  audits  will 
determine  whether  or  not  all  interfaces  between 
software  elements  and  between  software  and  hard¬ 
ware  have  been  identified  and  properly  defined. 


Again,  this  is  a  major  QA  concern  because  if 
interface  definition  is  ignored  in  the  Analysis 
and  Design  phases  of  the  development  effort,  there 
may  be  difficulties  when  we  try  to  compile  and 
test  the  software.  The  third  consideration  is  to 
ensure  that  the  design  is  proceeding  according  to 
the  baselines  established  at  the  previous  develop¬ 
ment  milestone.  That  is,  we  check  the  design 
specifications  against  the  system  requirements, 
the  product  specification  against  the  design 
specification,  and  the  coding  against  the  product 
specification.  Finally,  the  reviews  and  audits 
will  also  ensure  that  the  documentation  conforms 
to  the  established  standards  for  format  and  con¬ 
tent. 

The  QA  tasks  and  the  development  process  we 
have  briefly  outlined  in  this  paper  are  intended 
primarily  for  large  scale  design  efforts  and  must 
be  tailored  to  meet  the  specific  problems  often 
encountered  in  trainer  system  design  efforts. 
Characteristic  problems  of  trainer  system  develop¬ 
ment  are  shortened  schedules,  associated  data  re¬ 
quirements,  and  the  application  of  specifications 
more  suitable  to  development  of  critical  systems 
or  larger  systems  than  to  development  of  trainer 
systems.  Managers  and  engineers  experienced  in 
trainer  design  may  have  asked  questions  similar  to 
these:  How  can  we  write  design  specifications 

before  the  preliminary  design  review  when  it  is 
scheduled  two  months  after  contract  award?  Is  it 
cost  effective  to  write  design  specifications  if 
they  are  not  required  by  the  contract?  How  much 
of  the  software  control  exercised  for  critical 
systems  is  cost  effective  when  applied  to 
trainers? 

Often  trainer  development  schedules  are 
shortened,  especially  when  the  trainer  is  intended 
to  support  a  new  device,  because  trainer  design 
cannot  begin  until  che  device  itself  is  designed 
because  essential  design  data  must  be  available, 
but  the  customer  wants  the  trainer  delivered  in 
time  to  support  training  of  personnel  to  use  the 
new  device.  The  optimum  amount  of  time  required 
to  design  a  trainer  is  therefore  squeezed  into  a 
shorter  time  frame.  If  schedules  become  shortened 
to  the  extent  that  tasks  essential  to  the  software 
design  process  may  be  limited  or  deleted,  the 
impact  on  cost  and  delivery  must  be  assessed.  For 
example,  a  suggestion  to  skip  the  writing  of 
design  specifications  or  to  eliminate  module 
testing  must  be  evaluated  against  the  cost  which 
may  be  incurred  because  traceability  of  the  design 
was  lost  or  some  critical  modules  were  poorly 
designed  and  not  checked.  Conversely,  the  cost 
and  schedule  risk  of  having  to  redesign  software 
during  customer  acceptance  testing  may  justify 
allowing  more  time  at  the  beginning  of  the  program 
to  complete  all  design  phases  and  QA  tasks.  It 
should  be  a  function  of  the  Analysis  phase  to 
determine  whether  contract  requirements  are  con¬ 
sistent  with  schedule  constraints,  and.  If  there 
are  conflicts,  resolve  them  with  the  customer 
early  in  the  program.  A  more  risky  approach  would 
be  to  begin  software  development,  that  is,  require¬ 
ment  analysis  and  the  writing  of  design  specifi¬ 
cations,  prior  to  contract  award.  In  any  case.  If 
schedule  brevity  Is  a  problem,  it  must  be  ad¬ 
dressed  to  ensure  that  the  software  will  be  de¬ 
signed  correctly  and  on  schedule.  In  general, 
software  design  tasks  and  software  QA  tasks  should 
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not  be  eliminated  because  design  deficiencies  that 
are  not  discovered  early  in  the  design  process  are 
more  costly  to  correct  during  the  test  phase. 

Another  problem  which  might  be  unique  to 
trainer  system  development  is  abbreviated  data 
requirements.  What  is  meant  by  this  is  that  the 
customer  often  does  not  order  a  complete  set  of 
software  documentation  or  will  specify  documen¬ 
tation  that  attempts  to  combine  two  or  more  func¬ 
tions  into  a  single  document.  An  example  of  the 
first  case  is  a  contract  which  requires  Computer 
Program  Product  Specifications  but  does  not  re¬ 
quire  design  specifications.  Or,  rather  than 
ordering  both  design  and  product  specifications, 
the  customer  may  order  a  document  that  combines 
the  design  and  product  specifications  into  one 
specification.  In  either  instance,  what  might 
have  been  an  attempt  to  save  money  by  ordering  one 
less  document  does  not  .save  the  trainer  developer 
a  penny.  The  reason  for  this  is  that  to  develop 
reliable  software,  the  trainer  system  contractor 
to  write  both  specifications.  For  example,  if  a 
contract  calls  only  for  final  documentation,  the 
management  and  QA  objectives  which  are  dependent 
on  the  visiblity  provided  by  design  documentation 
will  be  difficult,  if  not  impossible,  to  meet. 
Rather  than  risk  losing  control  of  the  software 
design,  the  contractor  will  develop  some  means  of 
assuring  the  quality  of  tne  design  and  often  this 
concern  will  result  in  some  form  of  preliminary 
documentation  which  can  be  used  to  evaluate  the 
design  before  it  is  released  to  programmers  and 
finally  to  coding.  This  implies,  of  course,  that 
in  these  instances  the  contractor  will  generate 
documentation  for  which  he  will  not  be  directly 
paid.  The  alternative  approach  would  be  tne  "big 
bang"  theory  whereby  the  design  is  minimumly 
controlled,  if  at  all,  and  the  entire  software 
system  is  loaded  and  debugged  during  the  test 
phase  and  the  final  documentation  is  generated 
sometime  later.  Although  this  seems  to  be  a 
relatively  inexpensive  approach  compared  to  writ¬ 
ing  specifications  or  similar  documents  according 
to  defined  design  milestones,  studies  have  shown 
that  this  is  not  the  case.  In  fact,  it  has  been 
concern  over  the  high  cost  of  test  and  integration 
that  has  led  many  companies  to  develop  software 
management  and  QA  procedures  that  emphasize  main¬ 
taining  control  and  visibility  of  the  software 
during  the  entire  design  process. 

Instead  of  leaving  the  question  of  software 
QA  to  the  discretion  of  the  manufacturer,  the 
trainer  customer  might  decide  to  impose  software 
QA  requirements.  Often,  however,  the  requirements 
imposed  on  the  training  system  will  be  taken 
directly  from  the  requirements  imposed  on  the 
device  that  the  trainer  will  be  designed  to  simu¬ 
late.  This  may  result  in  what  could  be  termed  a 
software  QA  overkill.  For  example,  the  customer 
might  desire  a  training  system  that  corresponds  to 
a  new  fighter  aircraft,  which,  because  of  the 
criticality  of  the  system,  production  concerns, 
and  perhaps  the  handling  of  classified  data,  was 
developed  under  very  strict  software  management 
guidelines.  Usually,  these  take  the  form  of  more 
documentation,  more  design  milestones,  more  test¬ 
ing,  and  more  record  keeping.  The  training  sys¬ 
tem,  however,  will  probably  be  designed  to  operate 
in  a  classroom  environment  rather  than  at  super¬ 
sonic  speeds  in  the  upper  atmosphere,  will  Involve 


a  much  lower  production  run,  and  will  process  a 
smaller  amount  of  classified  data,  if  it  processes 
any  at  all.  To  impose  weapon  system  standards  on 
the  training  system  would  not  seem  to  be  cost 
effective  in  this  case,  and  could  result  in  a 
situation  where  the  cost  of  control  exceeds  the 
cost  of  the  software  itself.  What  is  needed  in 
instances  such  as  these  is  a  careful  analysis  of 
the  cost  and  objectives  of  the  software  system  to 
determine  the  level  of  control  necessary  to  ensure 
that  management  concerns  will  be  satisfied  through 
all  phases  of  system  development.  Hopefully,  a 
dialogue  between  the  contractor  and  the  buyer  will 
result  in  an  approach  that  balances  software  QA 
cost  and  software  performance  for  a  qiven  traininq 
system. 

It  is  clear  from  our  discussion  of  some  of 
the  problems  encountered  in  training  system  deve¬ 
lopment  that  software  management  and  QA  programs 
and  techniques  proven  in  other  areas  of  industry 
must  often  be  tailored  to  meet  the  needs  of  train¬ 
er  manufacturers.  One  way  to  approach  the  ques¬ 
tion  of  software  management  and  QA  from  a  company 
standpoint  would  be  to  come  up  with  a  suitable 
in-house  program.  In  other  words,  rather  than 
simply  responding  to  whatever  software  management 
and  QA  requirements  are  imposed  by  prospective 
buyers,  the  trainer  contractor  could  develop  an 
in-house  program  that  fulfills  the  objectives  of 
software  management.  The  advantage  of  this  ap¬ 
proach  is  that  once  the  in-house  software  manage¬ 
ment  and  QA  programs  are  off  the  ground,  unique 
customer  requirements  may  only  imply  a  tweaking  of 
the  working  system  rather  than  the  writing  of 
wholly  new  company  procedures.  Also,  if  the 
company  program  is  structured  to  meet  the  intent 
of  military  software  standards,  the  contractor 
will  probably  not  have  to  revise  any  of  the  in¬ 
ternal  procedures  to  meet  the  software  management 
and  QA  program  requirements  of  most  military  and 
commercial  customers. 

Assuming  a  decision  is  made  to  go  ahead  with 
developing  company  software  management  and  QA 
programs,  there  are  a  number  of  considerations 
which  must  be  addressed  according  to  peculiarities 
of  each  company  and  its  respective  business. 
Among  these  considerations  are  what  existing  model 
can  provide  the  basis  for  the  company  programs, 
how  much  control  should  be  exercised,  who  should 
be  assigned  to  the  various  tasks,  and  whether 
software  management  and  QA  should  be  handled  on  a 
company-wide  or  program  by  program  basis. 

As  we  have  already  implied  in  this  discus¬ 
sion,  the  existing  military  standards  and  speci¬ 
fications  probably  provide  the  best  model  for 
developing  software  manage, nent  and  QA  programs 
because  they  have  already  been  applied  in  many 
areas  of  industry  and  because  by  applying  such  a 
model,  we  could  satisfy  both  military  and  commer¬ 
cial  customers.  Since  much  of  the  instructive 
literature  on  the  subject  of  software  management 
and  QA  addresses  the  two  part  specification  ap¬ 
proach  as  a  management  tool  and  the  guidelins  of 
MIL-S-52779  as  a  QA  model,  if  a  company  decides  to 
use  this  approach  or  a  derivative  approach,  there 
are  plenty  of  helpful  hints  available.  Such  an 
approach  is  also  consistent  with  the  spirit  of 
standardization  because  if  all  contractors  deve¬ 
loped  unique  methods  of  software  design  and  docu¬ 
mentation,  communication  would  be  difficult. 
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Also,  It  Is  much  easier  to  explain  your  approach 
to  a  customer  if  it  sounds  familiar  than  if  it  is 
a  unique  system.  Another  source  for  software 
management  and  QA  models  would  be  the  systems 
employed  by  other  Industries,  of  which  there  are 
also  numerous  published  descriptions.  In  many 
cases,  however,  the  titles  used  to  describe  the 
various  design  phases  and  milestone  documents 
vary,  but  the  general  concepts  of  software  man¬ 
agement  are  very  similar  and  resemble  the  military 
model . 

Whatever  model  is  chosen,  a  decision  must  be 
made  to  determine  how  much  control  should  be 
exercised  given  the  cost  of  control  and  the  level 
of  performance  desired.  In  general,  the  software 
development  procedures  must  provide  enough  visi¬ 
bility  of  the  design  that  the  software  QA  reviews 
and  audits  can  be  accomplished.  Requirement 
traceability,  interface  definition,  configuration 
control,  and  documentation  standards  are  elements 
which  should  be  part  of  any  software  management 
program  and,  as  a  minimum,  the  software  QA  program 
should  include  sufficient  reviews  and  audits  to 
ensure  that  these  elements  are  present  in  each 
design.  Documentation  reviews  can  be  used  to 
check  traceability,  make  sure  that  all  interfaces 
are  defined,  and  verify  that  the  design  is  com¬ 
plete.  Participation  of  the  software  QA  repre¬ 
sentative  in  design  reviews  will  provide  an  oppor¬ 
tunity  for  questioning  unclear  aspects  of  the 
design.  Software  library  controls  and  periodic 
audits  of  test  reports  and  software  change  notice 
incorporaton  can  be  used  to  verify  that  configura¬ 
tion  control  is  maintained.  Since  it  may  not  be 
feasible  to  review  each  software  document  in 
detail,  sampling  inspections  may  be  planned  to 
ensure  that  the  established  design  procedures  are 
being  followed.  Reviews  of  corrective  action  and 
analysis  of  error  trends  are  useful  in  that  they 
provide  a  basis  for  determining  the  effectiveness 
of  software  management  efforts  and  recommending 
changes  to  standard  procedures.  Trend  analysis, 
especially  if  a  dollar  figure  can  be  applied,  can 
prove  valuable  for  the  software  QA  effort  because 
the  effectiveness  of  such  a  program  would  then  be 
expressed  in  dollars  saved.  This  analysis  could 
also  show  how  well  or  how  poorly  the  company 
software  management  procedures  compare  to  pub¬ 
lished  studies. 

While  making  a  determination  of  how  much 
control  should  be  exercised,  it  might  become 
evident  that  we  already  have  more  control  of 
software  development  than  we  realize.  For  ex¬ 
ample,  we  could  have  determined  that  the  software 
design  effort  was  out  of  control  because  no  docu¬ 
mentation  requirements  existed  in  the  company,  but 
found  that  most  of  the  designers  kept  some  form  of 
notebook  anyway.  This  would  not  be  an  optimum 
situation  from  a  management  or  software  quality 
point  of  view,  but  it  is  better  than  if  nothing 
was  written  down  at  all.  Perhaps  the  lead  engi¬ 
neers  had  ‘aken  it  upon  themselves  to  insist  that 
all  designers  keep  notebooks  and  thus  filled  a 
void  they  -ecognized.  What  we  are  Implying  here 
is  that  ail  software  QA  functions  do  not  neces¬ 
sarily  have  to  be  accomplished  by  dedicated  QA 
personnel  as  long  as  all  the  necessary  functions 
are  assigned  to  someone.  In  many  companies  a 
software  QA  department  has  been  created  and 
staffed  with  software  QA  experts,  while  in  other 


companies  certain  tasks  are  left  entirely  to  other 
departments,  such  as  engineering  or  configuration 
management,  or  assigned  to  other  departments  and 
then  monitored  by  the  software  QA  representatives. 
Library  controls,  for  example,  might  be  the  res¬ 
ponsibility  of  the  software  engineering  department 
but  subjected  to  periodic  QA  audits.  Or,  all 
configuration  management  functions,  such  as  soft¬ 
ware  change  control,  might  be  assigned  to  a  separ¬ 
ate  department  entirely  and  this  department  would 
also  be  responsible  for  configuration  reviews  and 
audits.  It  should  be  pointed  out  that  MIL-S-52779 
does  not  specify  that  all  QA  functions  be  per¬ 
formed  by  a  QA  department,  provided  all  functions 
are  performed. 

Whether  software  QA  functions  are  performed 
on  a  company-wide  or  a  program  by  program  basis  is 
also  left  to  the  discretion  of  the  contractor. 
This  determination  depends  in  part  upon  how  much 
control  is  desired  for  systems  that  have  no  soft¬ 
ware  QA  or  management  requirements  imposed  by  the 
customer.  Generally,  it  would  seem  desirable  to 
have  a  standard  set  of  procedures  to  fall  back  on 
if  none  were  imposed.  Some  of  the  functions, 
however,  could  be  handled  on  a  program  by  program 
basis  if  this  approach  seemed  most  cost  effective. 
For  example,  rather  than  establishing  a  central 
library  for  the  entire  company,  individual  soft¬ 
ware  libraries  could  be  controlled  by  each  program 
office.  In  this  case  we  would  have  a  company 
procedure  which  would  assign  responsibility  for 
library  control  to  the  program  offices.  The  same 
could  be  done  for  all  other  software  QA  functions 
and  there  could  even  be  a  ,*rson  assigned  to  all 
these  tasks  from  within  the  program  office.  Of 
course,  during  schedule  squeezes,  the  loyalty  of 
the  QA  representative  to  the  program  office  might 
compromise  the  objectives  we  set  out  to  accom¬ 
plish.  For  every  function  assigned  outside  the  QA 
department,  there  is  a  corresponding  risk  that  the 
individual  performing  a  quality  task  is  likely  to 
please  whoever  signs  his  paycheck,  whether  it  be 
QA  or  the  program  office.  Whatever  approach  is 
decided  upon,  it  is  Important  that  it  be  docu¬ 
mented  in  a  set  of  established  procedures  so  that 
all  personnel  involved  in  the  software  development 
effort  will  know  their  assigned  responsiblities. 

Obviously,  there  are  no  solutions  which  will 
work  for  all  companies  across  all  training  system 
development  projects.  It  is  up  to  each  manu¬ 
facturer  to  address  the  question  of  software 
management  and  QA  by  taking  into  account  the 
specific  problems  of  the  trainer  industry  and  the 
unique  management  objectives  within  the  company. 
The  existing  military  standards  and  specifications 
provide  a  model  which  can  be  analyzed  and  tailored 
to  meet  the  needs  of  a  company  program.  Perhaps, 
once  software  control  has  been  addressed  on  a 
company  by  company  basis,  a  set  of  software  stan¬ 
dards  applicable  to  the  trainer  industry,  coor¬ 
dinated  and  approved  by  all  industry  representa¬ 
tives,  might  be  written.  This  would  provide  the 
final  solution  to  most  of  the  questions  addressed 
by  this  paper. 
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ABSTRACT 


A  Software  Support  Facility  (SSF)  designed  initially  for  the  A-6E  Weapon  System  Trainer 
(WST)  and  located  at  the  Naval  Training  Equipment  Center  (NAVTRAEQUIPCEN) ,  Orlando,  is  seen 
as  a  significant  advance  in  capability  for  updating  software-intense  trainers.  This  paper 
describes  a  software  support  environment  that  facilitates  the  modifications,  testing,  and 
baseline  configuration  management  of  software  used  in  sophisticated  trainers.  The  software 
support  environment  consolidates  a  variety  of  techniques  into  an  effective  capability  for 
managing  the  implementation  of  modifications.  Baseline  configuration  management  is  founded 
on  the  implementation  of  operational  procedures  that  make  it  easy  to  adhere  to  configuration 
guidelines  but  difficult  to  get  changes  into  the  system  by  other  means.  Automated  identific¬ 
ation  of  global  symbol  generation  and  usage  greatly  reduces  risk  of  unforeseen  change  impact 
on  other  routines.  A  trainer  software  structure  is  described  that  permits  modification  of 
source  code  without  patches  and  without  the  normal  complete  link-edit  procedure.  A  conmun- 
ication  link  facilitates  rapid  turnaround  in  the  modification  and  test  cycle.  Potential 
support  of  trainers  with  different  computers  is  identified.  Further,  the  SSF  reality  makes 
possible  the  overall  integrated  environment  of  "man"  and  "machine"  for  management,  control 
and  visibility  for  software  support. 


INTRODUCTION 

The  maintenance  of  software  or  "software 
support"  is  an  ever  increasing  necessity  as  the 
complexity  and  the  number  of  software-intense 
trainers  grow.  Consistancy  between  these  emergent 
trainers  and  their  corresponding  parent  operation¬ 
al  systems  require  support  procedures  that  pro¬ 
vide  both,  rapid  and  accurate  update  methodology. 
Digital  computer  software  support  for  training 
devices,  therefore,  is  to  be  accomplished  through 
a  field  network  of  engineers,  progranmers  and 
technicians  utilizing  telephone  data  transmissions 
and  engineering  facilities.  The  Software  Support 
Facility,  designed  initially  for  the  A-6E  Weapon 
System  Trainer  (WST)  (See  Figure  1),  forms  the 
nucleus  of  the  software  support  concept. 

The  Software  Support  Facility  (SSF  is  loc¬ 
ated  at  the  Naval  Training  Equipment  Center  (NTEC) 
Annex  at  Herndon,  Orlando,  Florida.  Funded  by 
NAVAIR,  the  SSF  marks  a  first  at  NTEC  for  provid¬ 
ing  an  in-house  capability  for  its  software  sup¬ 
port  of  a  major  trainer.  Its  dedication  signifies 
recognition  that  support  of  trainer  software  nec¬ 
essitates  the  establishment  of  a  "computer  enviro¬ 
nment"  that  can  facilitate  the  modification,  test¬ 
ing  and  the  management  of  software  packages  resid¬ 
ent  on  current  trainers.  This  environment  can  be 
extended  to  support,  by  judicious  facility  growth 
in  accordance  with  the  added  workload,  many  of  the 
software-embedded  training  devices  planned  for,  or 
existing,  in  the  NAVY  Inventory. 

The  new  Software  Support  Facility  or  "SSF” 
is  an  outgrowth  of  the  Software  Development  Facil¬ 
ity  created  by  Grunnan  Aerospace  Corporation.  (1) 
This  recent  innovation  is  designed  to  centralize 


the  software  support  function  and  to  act  as  the 
major  element  in  servicing  and  supporting  training 
devices  located  in  multiple  remote  sites.  Note¬ 
worthy  is  the  establishment  of  the  communication 
links  between  NTEC  and  the  trainer  sites  at  NAS 
Oceann,  Virginia,  and  NAS  Whidbey  Island,  Wash,  via 
Remote  Job  Entry  (RJE)  and  time-share  terminals. 
These  RJE  terminals  make  possible  the  centralizat¬ 
ion  of  an  SSF  and  allow  the  utilization  of  the 
combined  resources  at  remote  sites  as  well  as  at 
the  SSF  to  accomplish  the  necessary  rapid  soft¬ 
ware  changes.  This  centralization  philosophy  pro¬ 
motes  a  more  cost-effective  solution  to  the  soft¬ 
ware  support  function  and  forms  the  catalyst  for 
the  support  approach  to  all  digital  computer 
driven  training  devices.  This  is  made  possible  by 
the  availability  of  Software  Management  Tools  to 
facilitate  systematic  software  modifications  coupl¬ 
ed  with  a  methodical  procedure  to  perform  the  soft¬ 
ware  accountability  or  "configuration  management" 
function  over  trainer  software.  These  tools,  work¬ 
ing  in  conjunction  with  a  file  oriented  data  base, 
provide  flexible  report  generation,  software  modif¬ 
ication  tracking  and  software  configuration  contrd. 
The  "software  tools,"  for  example,  allow  the  auto¬ 
mation  of  the  process  for  modifying  source  code 
(without  the  need  for  patches  to  executable  code) 
and  the  compilation  and  transmission  of  only  the 
affected  segment  of  the  software;  this  segment  is 
then  installed  at  the  site  without  the  necessity 
of  the  typical  complete  link-edit  procedure. 

From  the  nucleus  of  this  A-6E/WST  SSF,  extended 
software  support  is  planned  for  existing  and  future 
digitally  based  trainer  systems.  This  extension 
implies  that  continuity  of  SSF  operating  procedures 
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Figure  1 .  A-bE/WST 


persist  as  additional  digital  host  computers  are 
annexed  to  the  SSF.  As  long  as  the  software 
support  can  be  implemented  with  as  few  differences 
(computer  languages,  etc.)  as  possible,  the  person¬ 
nel  requirements  for  this  centralized  software 
support  Cfn  be  minimized. 

SOFTWARE  SUPPORT  OBJECTIVES 

The  evolution  of  centralized  software  support 
has  been  motivated  strongly  by  the  realization  that 
software  contributes  70-80%  of?the  life  cycle  cost 
of  current  computer  systems.  Further,  it  is 
recognized  that  computer  embedded  systems  (i.e., 
trainers,  etc.)  that  remain  in  the  inventory  for 
long  periods  (5-10  yrs)  also  follow  this  same  trend 
in  resource  allocation.  Concurrently,  the  flexi¬ 
bility  Inherent  in  software-intense  devices  neces¬ 
sitate  an  "automated  environment"  to  let  the  com¬ 
puter  handle  procedural  functions  automatically 
(i.e.  computations,  assemblies,  memory  management, 
etc.),  while  progranmer  modification  involvement 
is  minimized  to  source  code  charge  of  the  part¬ 
icular  trainer  program  segment  and/or  module. 

Under  these  considerations,  the  objective  of 
the  Software  Support  Facility  is  to  produce  a  high¬ 
ly  automated  and  Integrated  environment,  in  which 
the  centralized  software  support  function  can 
provide  service  to  a  variety  of  devices.  Central¬ 
ization  of  this  function  implies  that,  in  most 
cases,  the  training  device  will  not  reside  at  the 
SSF.  The  hardware  configuration  and  its  attendant 
data  communication  network  at  the  SSF,  therefore, 
provide,  (1)  rapid  and  accurate  data  transfer 
capability  for  modified  trainer  program  segments 


through  Remote  Job  Entry  (RJE)  terminals,  and  (2) 
trainer  site  time-share  terminal  access  to 
"baseline"  source  code  that  is  maintained  at  the 
SSF.  These  stipulations  fuel  the  fire  for  a  prov¬ 
ision  that  rigorous  "configuration  management"  or 
accounting  software  baseline  be  centrally  control¬ 
led  at  the  SSF.  At  the  same  time  provisions  for  a 
single  common  source  code  "baseline"  for  both  SSF 
and  trainer  site  personnel  is  available  for  train¬ 
er  software  modifications  -  the  best  of  both 
worlds . . . 

In  developing  such  a  facility,  goals  have  been 
selected  to  reduce  the  cost  of  development  and 
support  through  centralization.  This  provides  the 
opportunity  to  mechanize  each  trainer  software 
"baseline"  under  a  controlled  configuration  manage¬ 
ment  procedure  and  to  selectively  control  and  en¬ 
hance  the  value  of  the  companion  documentation. 
Subsidiary  goals  for  life-cycle  visibility,  trace¬ 
ability  and  manageability  are  fallouts  of  the 
centtalized  configuration  management  implementa¬ 
tion.  Finally,  the  SSF  computer  configuration 
coupled  with  the  trainer  software  "baseline", 
provides  a  known  departure  point,  or  platform  for 
technological  enhancements  or  growth  whether  it  is 
accomplished  in-house  or  contractually. 

The  SSF  has  been  designed  initially  for  A-6E/ 
WST  software  support  and  by  design  follows  the 
above  guidelines.  The  practices  and  procedures  are 
common  to  most  training  devices.  Additional 
devices  are  anticipated  to,  one-by-one,  join  the 
A-6E/WST  in  the  SSF  environment,  thereby  reducing 
substantially  the  cost  of  life-cycle  support  for 
trainers.  This  is  based  on  the  continuing  require- 
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meat  caused  by  update  and  change  of  operational 
equipment,  enhancements  to  training  or  operational 
value,  additional  capabilities,  system  Improvements 
for  stability,  error  recovery  or  upgrading  and 
finally,  correction  of  deficiencies. 

The  balance  between  responsive  trainer  soft¬ 
ware  support  and  cost-effective  operation  remains 
as  the  central  issue  in  the  SSF  Implementation. 

The  SSF  conf iguration  and  its  environment  are 
described  next. 

SUPPORT  ENVIRONMENT 

The  SSF  environment  Is  a  combination  of  digit¬ 
al  computer,  communication  devices,  and  a  set  of 
Software  Management  Utilities  or  "Tools"  to  auto¬ 
mate  many  of  the  pedestrian  procedures  necessary 
to  edit  and  construct  a  trainer  program.  The 
operating  procedures  mold  this  assortment  of  enti¬ 
ties  into  a  cohesive  and  cost-effective  support 
system. 

Hardware  Configuration 

The  facility  currently  consists  of  a  Perkin- 
Elmer  8/32  digital  computer,  4  disc  drives,  2  tape 
units  and  associated  hardware  to  provide  direct 
communications  with  Oceana  and  Uhidbey  Island.  A 
Versatec  Printer/Plotter  is  used  for  making  hard 
copy  of  the  graphics  data  normally  directed  to  the 
Instructor  Console.  A  complement  of  approximately 
10  Navy  personnel ,  with  the  help  of  six  (6) 
employees  of  Grumman  Aerospace  Corporation  man  the 
facility  (See  Figure  2).  The  computer  hardware 
configuration  is  selectively  tuned  to  the  software 
support  mission.  Its  core  memory  of  one  million 


data  bytes  proyide  sufficient  space  to  accoamodate 
source  program  storage,  workload  capacity,  soft¬ 
ware  configuration  control,  tracking  and  utility 
programs  for  the  SSF  and  trainer  site  personnel  to 
efficiently  carry  out  their  mission. 

Communication  Network 

The  communications  network  currently  implement¬ 
ed  uses  dial-up  telephone  lines  to  provide  high  and 
low  haud  rate  capability  for  RJE  and  CRT  terminals, 
respectively  to  Oceania  NAS,  Va.  and  Whidbey 
Island  NAS,  Washington.  Figure  3  sumnarizes  the 
network  details,  while  Figure  4  shows  the  commun¬ 
ications  network  currently  in  place. 

Operational  Procedures 

The  hardware  configuration  provides  an  optimum 
mix  for  the  support  of  centralized  "software  con¬ 
figuration  management"  and  rapid  access  of  base¬ 
line  source  programs  via  either  SSF  or  field  time- 
share  terminals.  To  balance  the  workload  and  limit 
communication  transmission  durations,  an  operation¬ 
al  framework  between  the  SSF  and  field  sites  is 
necessary. 

An  account  structure  has  been  Installed  on  the 
SSF  computer  system  to  Implement  this  operational 
environment.  A  "system"  account  heads  the  hier¬ 
archical  account  structure;  it  contains  such  pro¬ 
grams  as  Command  Substitution  System  (CSS)  proced¬ 
ures,  FORTRAN  compiler,  assembler,  RJE  communicator 
data  base  file  system  (TOTAL)  and  other  system 
software  utilities  to  properly  manage  the  varied 
workloads  resident  on  the  computer.  Directly  under 
the  "system"  account  are  "group"  accounts  which 
contain,  (1)  various  categories  of  trainer 
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Figure  3.  SSF  to  Site  Commmlcatlon  Capability 


"baseline"  source  programs,  (2)  software  tool 
programs,  and  (3)  software  system  component  pro¬ 
grams  (l.e.  I/O,  Graphics  pages,  etc.).  At  the 
third  level,  private  accounts  are  established  be¬ 
neath  the  system  and  group  accounts.  In  which  mod¬ 
ifications  are  carried  out  by  SSF  and  field  person¬ 
nel.  The  system  and  group  accounts  are  protected 
from  change  by  giving  access  through  password  to 
a  selected  few  In  the  SSF;  In  particular,  to  the 
conflgruat Ion  management  personnel.  Private 
accounts  can  read  from  accounts  above  In  hier¬ 
archy,  but  cannot  change  or  replace  programs  in 
these  accounts;  this  Is  reserved  for  the  config¬ 
uration  management  function. 

In  this  way,  modifications  can  be  made  In 
parallel  (private  accounts  have  access  to  source 
code  baseline)  and  submitted  to  configuration 
management  for  approval.  Configuration  manage¬ 
ment,  In  turn,  then  processes  these  modifications 
according  to  a  methodical  procedure.  Including 
Software  Review  Board  approval,  compilation  of  the 
new  module,  generation  of  a  new  task  segment  or 
"overlay"  (configuration  management  tools  are 
available  that  automate  much  of  this  procedure) 
and  the  transmission  of  the  "overlay"  to  the 
appropriate  trainer  sites  over  the  conmunlcation 
network.  Following  this  sequence,  the  site 
personnel  Install  this  overlay  on  the  designated 
trainer  and  carry  out  the  appropriate  test  pro¬ 
grams  to  verify  the  software  modification.  The 
final  step  is  then  for  the  configuration  manage¬ 
ment  personnel  to  update  the  software  baseline  at 


the  SSF  on  the  successful  completion  of  the 
trainer  test.  If  deficiencies  are  found  during 
the  test  sequence,  they  are  recorded  on  the  Test 
Discrepancy  Report  and  modification  continues 
until  all  deficiencies  are  corrected. 

Software  Management  Tools 

Salient  features  of  the  software  design  for 
the  A-6E/WST  were  selected  to  create  both  a  stable 
developittr.t  and  a  stable  support  environment.  This 
in  turn,  stabilized  the  procedures  and  yielded  a 
generalized  set  of  Software  Tools  that  could  be 
used  to  automate  many  of  the  configuration  manage¬ 
ment  and  modification  functions.  In  extending 
these  tools  to  future  trainer  support,  trainer 
software  design  features  that  are  desirable  to 
take  maximum  advantage  of  this  generalized  support 
environment  are  given  as  follows: 

(1)  Modularized  software  structure. 

(2)  Some  categorization  scheme  that  will 
permit  assignment  of  unique  Identification 
numbers  In  logical  manner. 

(3)  Descriptive  Information  on  each  module. 

(A)  Established  Mnemonic  convention  format 

for  symbols,  modules,  etc. 

(5)  Higher  Order  Language  (HOL) 

(6)  Partitioning  of  large-scale  programs  to 
Implement  source  modifications  that  will 
not  require  excessive  transmission  time 
to  site. 

The  Grumman  developed  Test  and  Configuration 


Figure  4.  SSF  Communication  Network 
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Figure  5.  TACT  System  Configuration 
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Tracking  System  (TACT)  is  a  file  oriented  data 
base  system  that  combines  a  technical  reporting 
system  with  a  management  information  system.  It 
is  supplemented  by  analysis  and  test  support  tools 
Chat  further  enhance  the  flexibility  for  data  base 
control,  and  for  the  analysis  of  "as-built"  code. 

The  TACT  system  (described  in  more  detail  in 
reference  1)  is  operated  in  two  modes,  (1)  on-line 
system,  with  interactive  control  for  quick-look 
inquires  and  data  base  update  and,  (2)  batch 
report,  hard  copy  processing.  Three  major  funct¬ 
ional  subsystems  makeup  the  TACT  system;  they  are: 

(1)  Software  Tracking  System  (STS) 

(2)  Interface  Tracking  System  (ITS) 

(3)  Test  Tracking  System  (TTS) 

The  TACT  system  utilizes  the  TOTAL  data  base  man¬ 
agement  system  marketed  by  CINCOM,  Inc.  The  TACT 
system  configuration  is  shown  in  Figure  5. 

The  software  tracked  by  the  TACT  system  is 
organized  into  a  hierarchical  structure  so  that 
each  subsystem/module  is  identified  with  a  unique 
Hierarchical  Identification  (HID)  code.  The  Soft¬ 
ware  Tracking  System  (STS)  is  used  to  collect  and 
track,  through  a  module  data  base,  various  para¬ 
meters  of  each  module.  Data  such  as  module  name, 
mode  usage,  execution  order,  execution  time,  core 
size  actuals,  input  and  output  mnemonic  names, 
version  level,  source  language  and  cognizant  eng¬ 
ineer,  are  a  few  of  many  variables  that  are  used 
to  describe  the  anatomy  of  each  module.  In  the 
support  role,  it  is  crucial  to  maintain  this  data 
base  through  the  various  modification  projects  for 
each  device.  The  STS  system  serves  the  role  of 
maintaining  this  fundemental  data  base  as  a  part 
of  the  software  configuration  baseline. 

The  Interface  Tracking  System  (ITS)  automates 
the  handling  of  module  interconnect  variables  and 
the  signal  variables  between  the  software  and  the 
trainer  hardware.  A  Data  Element  Dictionary  (DED) 
and  Signal  Description  List  (SDL)  are  examined  by 
subordinating  programs  that  supply  the  ITS  with 
current  Interface  data.  These  programs  basically 
assist  in  the  automatic  generation  of  all  the 
Interface  parameters  of  the  system.  The  ITS  plays 
a  significant  role  in  the  software  modification 
procedure  by  automatically  identifying,  for  a 
given  change  of  variable,  the  modules  and  parameters 
affected  by  that  change.  The  extent  or  the  re¬ 
sources  needed,  for  a  software  change  can  be  quick¬ 
ly  assessed  by  the  use  of  this  portion  of  TACT. 

An  ancillary  program.  Automatic  Interface  Mneuraonic 
Extractor  System  (AIMS),  is  used  to  systematically 
dissect  a  software  module  to  classify,  extract 
and  identify  the  attributes  (global  vs  local  or 
input  vs  output,  etc.)  of  module  variables  across 
a  spectrum  of  FORTRAN,  assembly  code  or  a  mixture 
thereof.  The  AIMS  builds  the  data  base  for  the 
ITS  program.  Through  this  medium  various  reports 
can  be  gathered  to  identify  missing  inputs,  unused 
outputs  or  other  pertinent  data  necessary  to  in¬ 
sure  the  integrity  of  a  modified  trainer  program. 

The  Test  Tracking  System  (TTS)  Is  an  inform¬ 
ation  management  system  which  maintains  a  data 
base  that  references  test  procedures  and  the 
resultant  Trainer  Discrepancy  Reports  (TDR).  The 
elements  maintained  are  the  current,  relevant  test 
procedures  by  module  or  by  mode,  the  responsible 
engineer,  subsystem  name,  description  of  discre¬ 
pancy  or  change  and  its  disposition.  Reports  are 


automatically  generated  (in  batch  mo  !c)  for  the 
day-to-day,  week-to-week  tracking  of  these  TDR's. 
They  are  to  be  used  by  the  various  levels  of  man¬ 
agement  for  resource  allocation,  for  failure  anal¬ 
ysis,  for  visibility  of  trouble  spots,  etc.  A 
"quick  look",  as  well  as,  update  of  TDR  data  base 
are  available  through  an  "on-line"  TTS  terminal. 

Access  to  'he  TACT  system  via  CRT  terminal 
from  the  trainer  sites  either  on-line  or  batch,  is 
provided  through  the  VADIC  modem  in  a  time-shared 
mode.  It,  in  fact,  gives  the  site  personnel 
access  to  the  appropriate  source  programs,  software 
tools,  compilers,  etc.,  that  reside  on  the  SSF. 
These  software  utilities  are  summarized  in 
Figure  6.  d) 

Various  software  tools  are  also  available  at 
the  SSF  to  maintain  the  baseline  conf iguration  of 
the  trainer  software.  The  procedure  for  the  maint¬ 
enance  of  the  software  baseline  is  a  methodical 
discipline  put  in  a  centralized  location  (for  more 
than  one  device  location),  to  track,  control  and 
record  the. history  of  change  for  a  given  trainer. 
Figure  7  1  '  illustrates  the  steps  necessary  to 
not  only  maintain  the  software  baseline,  but  also 
to  coordinate  the  configuration  updates  to  the 
documentation.  These  software  tools  (1)  provide 
complete  tracking  and  identification  of  all  auth¬ 
orized  changes,  (2)  include  header/trailer  programs 
to  emplace  configuration  identity  and  quality 
assurance  information  on  each  source  file,  (3)  in¬ 
clude  configuration  accounting  programs  for  list¬ 
ing  software  components  and  revision  levels  and 
for  automatically  extracting  configuration  data 
from  header/trailer  blocks,  and  (4)  generate  all 
load  modules  from  computerized  configuration  con¬ 
trol  records. 

The  automated  configuration  management  tools 
and  the  procedures  laid  out  In  Figure  7  are  leading 
toward  the  generation  of  an  overlay  which  is  ulti¬ 
mately  transmitted  to  the  trainer  site.  The  over¬ 
lay  is  a  program  segment  which  has  undergone  a 
modification  under  a  change  order,  recompiled  and 
structured  into  the  form  of  an  executable  load 
module  for  installation  on  the  appropriate  trainer. 
The  overlay  is  directed  to  a  specific  window  in 
memory  that  normally  contains  the  approved  revis¬ 
ion  level  of  code.  The  overlay  is  structured  to 
intercommunicate  with  other  overlays  via  Fortran 
Labeled  Common  which  avoids  the  need  to  link-edit 
the  entire  program.  This  automated  sequence  in¬ 
cludes  the  verification  of  compliance  to  the  base¬ 
line  configuration  prior  to  any  processing  of  a 
change  component.  It  concludes  with  the  accept¬ 
ance  testing  after  Installation  and  the  revision 
and  release  of  the  new  software  configuration 
baseline . 

EXTENDED  TRAINER  SUPPORT  PLAN 

The  rationale  for  centralizing  the  resources 
for  the  trainer  device  software  support  becomes 
stronger  as  the  number  of  training  devices  at 
different  locations  increase.  This  Is  substant¬ 
iated  by  a  recent  study  prepared  by  Computer 
Sciences  Corporation  for  NAVTRAEQU I PCEN  Code  N-412, 
Orlando,  Florida. 

Short-Term  Extension  of  SSF  Capability 

it,) 

Under  this  plan,  SSF-conf lgured  host 
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Figure  7.  Configuration  Management  Procedure 
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Figure  8.  Proposed  Orlando  Software  Support  Expansion 


computers  (supporting  an  additional  and  appropriate 
segment  of  NAVY  training  devices)  would  sequent¬ 
ially  be  added  to  the  existing  SSF.  This  expanded 
facility  is  shown  in  Figure  8.  The  management 
tools  already  described  are  Co  be  modified  and  be 
compatible  with  these  additional  host  computers; 
the  communications  network  would  also  be  expanded 
to  three  main  dedicated  telephone  circuits  (shown 
in  Figure  9)  to  service  most  of  the  Naval  training 
facilities  within  the  CONUS. 

The  procedures  and  practices  now  implemented 
at  the  SSF  are  seen  to  be  conmon  to  the  expanded 
SSF.  This  commonality,  in  turn,  will  provide  a 
manpower  cost-effective  base  on  which  to  operate 
the  centralized  facility. 

Long-Term  SSF  Implementation  Coal 

Greatly  improved  performance  with  minimum 
duplication  of  both  effort  and  hardvare  may  event¬ 
ually  be  provided  through  the  restructuring  of  the 
Software  Support  Facility  to  include  a  mainframe 
computer.  In  this  configuration  many  of  the  soft¬ 
ware  tools  would  be  combined  and  Implemented  on 
the  mainframe  computer  ,  thereby  eliminating  the 
need  Both  for  supporting  multiple  implementation 
of  all  the  tools  on  separate  minicomputers  at  the 
SSF  and  for  extending  implementation  of  those  tools 
when  additional  minicomputers  are  brought  Into  the 
SSF.  The  mainframe  computer  vould  be  interconnect¬ 


ed  with  minicomputers  within  the  SSF  to  provide  a 
capability  for  using  compilers,  assemblers,  link- 
editors  and  utility  packages  from  the  OEM  of  the 
target  computers.  In  some  cases  emulation  of 
target  computers  would  be  used  so  that  certain 
computers  would  not  have  to  be  maintained  within 
the  SSF  environment.  Conmunlcatlon  links  would 
connect  with  the  mainframe  computer  rather  than 
with  each  separate  minicomputer.  The  mainframe 
would  direct  data  to  a  minicomputer  for  compilat¬ 
ion,  assemblies,  or  link-edit  as  appropriate  and 
receive  the  results  for  storage.  Minimal  auxil¬ 
ary  storage  would  be  dedicated  to  each  minicomputer 
for  working  storage. 

The  implementation  thus  described  would  great¬ 
ly  facilitate  operations  both  local  and  from  the 
field.  Locally  there  would  b„  -educed  duplication 
of  equipment  and  procedures  amo>.6  the  numerous 
computer  systems  involved.  The  consolidation  of 
trainer  configuration  baseline  management  into  a 
mainframe  computer  not  only  reduces  the  need  to 
learn  multiple  systems  but  also  tends  to  provide 
opportunities  for  global  analysis  of  multiple  base¬ 
lines  across  numerous  target  computers.  The  modif¬ 
ication  efforts,  both  local  and  field-based,  can 
be  implemented  using  a  single  text  editor  and 
common  operating  procedures  regardless  of  the 
Identity  of  the  target  computer  involved.  The 
primary  aspect  of  modification  that  cannot  practic¬ 
ally  be  made  transparent  to  the  user  are  the 
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language  difference*  and  the  software  testing 
efforts  associated  with  the  different  target 
computers. 


The  benefits  of  establishing  a  centralized 
Softvare  Support  Facility  for  use  by  local  and 
field  modification  efforts  rise  to  their  maximum 
potential  with  the  implementation  of  software 
tools  on  a  centralized  mainframe  computer. 


SUMMARY 


The  centralized  SSF  environment  provides  a 
cost-effective  method  for  the  implementation  of 
softvare  support  to  training  devices  widely  dis¬ 
tributed  in  location.  It  enhances  the  use  of  key 
software  skills,  encourages  software  commonality, 
makes  use  of  common  software  maintenance  tools  and 
standards  and  reduces  redundant  support  hardware. 


This  centralized  facility  provides  the  soft¬ 
ware  baseline  control  or  configuration  management 
function  that  for  so  long  has  been  missing.  It 
accomplishes  this  while  allowing  the  primary 
device  to  function  on  its  Intended  mission.  The 
flexible  communication  network  allows  maximum 
accessibility  to  authorized  NAVTRAEQUIPCEN  field 
personnel  so  that  the  most  knowledgeable  people 
on  trainer  operation  can  be  used  to  diagnose 
problem  areas.  All  the  power  of  the  SSF  environ¬ 
ment  is  virtually  at  t  ie  disposal  of  the  field 
personnel . 


Incremental  change  and  growth  are  a  way  of 
life  for  large  software  systems.  Design  evaluat¬ 
ion,  aircraft  updates  and  changes,  and  the  addit¬ 
ion  of  features  to  enhance  training  value  require 
a  well-coordinated  and  automated  software  support 
environment  like  that  available  at  the  SSF.  The 
SSF  also  serves  to  reduce  the  cost  of  these 
functions  and  Increase  the  life-cycle  traceability 
and  manageability  of  the  software  support;  the 
direct  accessibility  to  source  programs,  process¬ 
ors,  tools  and  documentation;  and  direct  data 
communication  to  the  device  sites. 


The  SSF  serves  as  a  "baseline"  for  extend¬ 
ing  central  software  support  to  other  trainers 
with  their  attendant  target  computers.  The  Soft¬ 
ware  Management  Tools  and  Procedures,  for  example, 
are  essentially  applicable  with  some  adjustments 
to  those  applications.  The  orchestration  of  the 
procedures  necessary  to  efficiently  deliver  this 
service  to  the  Fleet  is,  and  will  remain,  the 
primary  mission  of  the  Software  Support  Facility. 
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ABSTRACT 

As  budget  limitations  continue  and  shortages  of  experienced  military  personnel  persist, 
more  and  more  attention  is  being  given  to  alternative  support  concepts,  particularly  when 
they  offer  potential  cost  savings  or  cost  avoidance  while  meeting  equipment  availability 
requirements.  This  paper  examines  current  alternatives  to  conventional  training  device 
support  concepts  with  emphasis  on  Fixed-Price,  Penalty/ Incenti ve  Contractor  Support  (FPPICS). 


BACKGROUND 

The  Naval  Training  Equipment  Center  has  been 
providing  training  equipment  to  the  Navy  for  over 
three  decades.  In  that  time,  equipments  have 
undergone  a  dramatic  change  in  scope,  complexity 
and  cost.  The  current  Navy  training  equipment 
(COG  "20")  inventory  includes  devices  from  simple 
inexpensive  cutaways  to  state-of-the-art  air 
trainers  with  six  degrees  of  freedom  motion 
platforms,  computer  imagery  and  a  vast  array  of 
digital/hybrid  circuitry. 

The  current  value  of  the  inventory  of 
training  equipments,  and  the  near  term  projected 
acquisitions  will  push  the  value  of  this  equip¬ 
ment  and  that  of  other  services  into  the  billions 
of  dollars. 

As  expensive  as  the  acquisition  cost  of  this 
hardware  may  be,  the  cost  of  ownership  can  easily 
surpass  the  acquisition  cost.  When  one  considers 
owning  this  equipment  for  15  to  20  years,  the 
cost  of  ownership  becomes  staggering. 

The  Naval  Training  Equipment  Center  has 
supported  these  equipments  in  numerous  ways  over 
the  past  three  decades,  but  rather  than  present 
an  anthology  of  these  support  systems,  it  would 
be  more  appropriate  to  describe  the  current 
support  mechanism  as  it  exists  today  for  the 
majority  of  training  devices. 

Conventional  Support  Methods 

When  a  major  training  device  is  procured, 
the  Naval  Training  Equipment  Center  also  procures 
interim  support  from  the  prime  contractor  to 
allow  the  Government  support  system  time  to 
become  operational.  Interim  support  is  typically 
for  a  twelve-month  period  and  includes  an  on-site 
contractor  representative,  interim  spare  parts 
and  preliminary  documentation.  During  this 
interim  support  period,  the  contractor  is  fully 
responsible  for  ensuring  the  training  device  is 
operational . 

The  Navy  support  system  is  typically  composed 
of  Navy  Training  Device  Technicians  (TRADEVSMEN- 
TD’s)  to  support  the  training  unique  equipment, 
and  fleet  experienced  personnel  to  support  opera¬ 
tional  Government  Furnished  Equipment  (GFE). 

These  technicians  may  include  Electronics  Tech¬ 
nicians  (ET's),  Sonar  Technicians  (ST's),  or 
Data  System  Technicians  (OS's)  to  name  only  a  few. 


The  fleet  experienced  personnel  normally 
need  little  or  no  training  since  they  have 
experience  on  the  operational  systems.  However, 
training  unique  equipment  has  no  fleet  operational 
counterpart  and  TD's  normally  undergo  extensive 
training  courses  in  operation  and  maintenance  of 
training  unique  equipment.  Since  the  percentaqe 
of  training  unique  ("simulated")  equipment  contin¬ 
ues  to  increase,  it  is  not  uncommon  for  these 
courses  to  extend  for  as  long  as  six  months  to  a 
year  in  length.  These  technicians  will  normally 
perform  the  preventive  and  corrective  maintenance 
at  the  organizational  and  intermediate  level,  with 
depot  level  repair  being  reserved  for  a  major 
support  facility  or  a  contractor. 

To  aid  the  Navy  technicians,  a  civilian  on¬ 
site  Government  technical  representative  called 
the  Field  Engineering  Representative  (FER)  is 
provided.  The  FER  is  the  on-site  technical  expert 
as  well  as  the  liaison  between  the  civilian  sup¬ 
port  agency  and  the  military  community. 

The  Naval  Training  Equipment  Center  procures  a 
myriad  of  technical  information  in  the  way  of 
drawings,  reports,  handbooks,  manuals  and  documen¬ 
tation  to  assist  these  individuals  in  the  mainten¬ 
ance  required  functions  support  to  the  device. 

The  costs  of  this  technical  documentation  has  been 
rising  in  spite  of  efforts  to  reduce  or  contain 
them  and  frequently  runs  into  the  millions  of  dol¬ 
lars. 

To  back  up  the  on-site  Government  support 
organization,  there  are  three  field  office  exten¬ 
sions  of  the  Naval  Training  Equipment  Center.  The 
field  organizations  are  located  in  San  Diego, 
California;  Norfolk,  Virginia;  and  Pensacola, 
Florida.  These  support  centers  provide  engineer¬ 
ing,  software,  and  maintenance/logistic  support  to 
the  device  sites. 

To  complete  the  support  program  of  the 
Government,  there  must  be  a  mechanism  to  supply 
and  replenish  parts.  As  mentioned  earlier,  the 
Government  procures  interim  spare  parts  to  be 
delivered  with  the  device.  At  a  later  date,  the 
Naval  Training  Equipment  Center  will  furnish  the 
using  command  with  funds  to  procure  Initial  Out¬ 
fitting  parts,  and  the  Aviation  Supply  Office, 
Philadelphia,  Pennsylvania,  will  procure  parts 
and  stock  them  at  a  supply  facility  remote  from 
the  device  sites. 
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This  then,  is  the  basic  structure  of  the 
conventional  Government  support  system  for 
training  devices.  While  experience  has  shown 
that  this  support  system  does  work  and  generally 
meets  training  needs,  there  are  many  inherent 
problems  associated  with  the  basic  nature  of  this 
system  that  prohibit  it  from  being  cost  effective 
over  the  life  cycle  of  the  device.  These  problems 
are  sunmarized  below: 

•  Training  device  technicians  ( TO ‘ s )  rotate 
from  device  to  device  every  2  to  3  years. 

•  The  large  majority  of  TD's  are  inexperi¬ 
enced  technicians. 

•  The  Navy  is  having  a  difficult  time 
retaining  experienced  maintenance  tech¬ 
nicians  in  today's  high  technology  envi¬ 
ronment. 

•  The  Government  receives  constant  pressure 
to  reduce  the  federal  billet  structure 
both  military  and  civilian. 

•  Electronics  state-of-the-art  is  constantly 
changing. 

•  TD's  as  a  rule  do  not  stay  in  the  same 
training  device  community  upon  transfer; 
i.e.,  air,  surface,  subsurface  devices, 
necessitating  even  longer  retraining 
courses. 

•  The  Aviation  Supply  Office  is  set  up  to 
support  multiple  unit  aircraft,  rather 
than  the  small  population  of  unique 
trainina  devices. 

•  The  field  support  organizations  are 
having  a  difficult  time  recruiting  and 
retaining  qualified  software  personnel. 

This  is  not  meant  to  be  a  comprehensive  list 
of  problems  but  rather  an  overview  of  the  more 
serious  problems.  With  this  support  scenario  in 
mind,  it  is  time  to  explore  other  more  cost 
effective  support  systems  for  training  devices. 

Experience  Summary 

Contractor  support  is  not  new.  The  Air  Force 
and  the  Navy  have  used  it  to  support  training 
aircraft  and  training  equipment  on  a  continuing 
basis  and  as  mentioned  above,  many  contracts 
contain  "interim  contractor  support"  provisions 
for  support  of  new  equipment  during  the  initial 
deployment  phase  while  the  government  capability 
for  support  is  being  acquired/developed, 

The  Naval  Training  Equipment  Center  as  a 
matter  of  course  routinely  contracts  for  "interim 
contractor  support"  to  allow  the  Government  suffi¬ 
cient  time  to  initiate  Government  support.  The 
interim  contractor  support  is  generally  one  year 
in  length,  buy  may  be  significantly  shorter  if  the 
nature  of  a  device  warrants  it. 

The  Government's  experience  with  interim 
contractor  support  has  shown  a  high  device 
availability  with  excellent  parts  support  from 
the  contractor.  In  general,  device  availability 
has  declined  after  the  end  of  the  Interim  contrac¬ 
tor  support  period  when  the  contractor  has  left 


the  site.  In  addition,  most  contractors  have 
experience  in  support  of  their  own  equipment.  At 
Gould  Inc.,  Simulation  Systems  Division,  support 
has  been  provided  across  the  full  spectrum  from 
conventional  support  to  fixed-price,  penalty/ 
Incentive  contractor  support  -  the  subject  of  this 
paper.  The  fixed-price,  penalty/incentive  concept 
has  been  used  on  the  T-34C  CPT  program  very  suc¬ 
cessfully.  It  is  also  planned  and  is  being  imple¬ 
mented  for  the  T-34C  FIT  and  T-44  OFT  programs. 

In  addition,  the  concept  has  been  proposed,  in 
many  cases  at  Government  request,  for  several 
recent  programs  including  the  F-18  Trainer  Family, 
the  Reactive  Electronic  Equipment  Simulator,  and 
the  MK-23/MK-86  Gun  Fire  Control  System  Simulators. 

ALTERNATIVES 

The  alternatives  examined  in  this  paper 
include  conventional  support,  Contract  Maintenance 
Services  (CMS)  support,  commercial  computer  sup¬ 
port,  fixed-price  support  without  penalty/incentive 
and  Fixed-Price,  Penalty/ Incentive  Contractor  Sup¬ 
port  (FPPICS).  Table  1  summarizes  these  alterna¬ 
tives  from  a  management  and  support  viewpoint  and 
is  referenced  throughout  the  following  discussion. 

Conventional  Support 

Conventional  support,  as  described  previously, 
is  the  most  frequently  used  concept  at  this  time. 
Pecause  of  the  extent  of  experience  with  this 
approach,  it  has  become  very  highly  defined  with 
limited  flexibility.  Recent  efforts  indicate  that 
some  flexibility  may  be  restored  to  this  approach 
by  encouraging  prospective  bidders  to  suggest 
alternative  methods  of  meeting  specific  support 
element  requirements  if  support  is  not  jeopardized 
and  the  requirements  in  questions  can  be  met  more 
cost  effectively.  However,  this  flexibility  has 
seen  very  limited  implementation  to  date. 

Contract  Maintenance  Services  (CMS)  Support 

Contract  Maintenance  Services  is  very  similar 
to  conventional  support.  It  is  normally  procured 
as  a  cost-plus  type  services  contract.  Essen¬ 
tially,  service  contracts  involve  the  contractor 
providing  support  personnel  in  lieu  of  uniformed 
or  Government  civil  service  personnel,  usually 
the  full  range  of  support  element  requirements 
remains  in  the  equipment  acquisition  contract. 
Therefore,  limited  costs  savings  potential  exists, 
primarily  from  a  lower  turnover  and  training  rate 
associated  with  contractor  personnel  and  from 
potentially  lower  contractor  manning  levels  com¬ 
pared  with  military  (approximately  0.75:1). 

Commercial  Computer  Support 

Commercial  computer  support  of  digital 
computers  by  contractor  maintenance  services  allows 
the  Government  the  flexibility  of  establishing 
conventional  support  for  much  of  the  training 
unique  portion  of  the  trainer,  while  having  the 
advantage  of  contractor  support  of  the  computer 
system.  Many  training  devices  are  using  state-of- 
the-art  high  speed  conmercial  digital  computers 
that  have  a  support  organization  already  in  place 
from  the  computer  manufacturer.  This  can  provide 
a  more  cost-effective  alternative  to  the  require¬ 
ment  to  provide  Government  technicians  with  the 
lengthy  trainig  necessary  for  maintenance  o* 
these  computers,  as  well  as  the  requirem*"'  •  >• 
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TABLE  1.  ALTERNATIVE  SUPPORT  CONCEPTS  -  COMPARATIVE  OVERVIEW 


ELEMENT/FACTOR 


Type  of  contract 
Risks: 

-  Cost  Liability 

-  Transition 


Availability 
(Operational ) 

Cost  Control 


Cost  Visibility 


Incentives 


Configuration 

Control 

Maintenance 
Management 
and  Site 
Feedback 


Maintenance/Repai 

Concept 


Logistic  Support 
Analysis 

Reliability/ 

Maintainability 

Technical 

Manuals 

Training 


Provisioning 


CONVENTIONAL  SUPPORT  VS.  CONTRACT  MAINTENANCE  SERVICES 
CONTRACT  MAINTENANCE 

CONVENTIONAL  SUPPORT  SERVICES  (CMS)  SUPPORT  _ _ 


COMMENTS 


MANAGEMENT  CONSIDERATIONS 


N/A 


Government  assumes 
responsibility. 

Minimal  -  up  front 
acquisition. 

Government  assumes 
responsibility. 

Government  assumes 
responsibility. 


Government  assumes 
responsibility. 

Conventional  contrac¬ 
tual  incentives. 

Standard  military 
procedures . 

Standard  military 
procedures  -  (3M 
system) . 


Maximum  funded  level 
of  effort. 


No  contractual  limit. 


Minimal  -  up  front 
acquisition. 

Goal  only. 


Open  ended-level  of 
effort  budgetary 
estimate. 

Contractual  reporting 
requirements. 

Conventional  contrac¬ 
tual  incentives. 

Standard  Configuration 
Management. 

Service  procedures  + 
contractor  procedures. 


Normally  either  cost  plus  or 
fixed-price  labor  with  cost  plus 
for  material. 

As  required  by  program. 


R/M  program  provides  some  confi¬ 
dence  of  inherent  equipment 
availability. 


Product  improvements  may  be 
recommended  including  cost  impact. 


LOGISTIC  SUPPORT  CONSIDERATIONS 


Per  maintenance 
analysis;  geared 
toward  government 
support. 

Styled  after  standard 
government  support. 

Tailored  to  contract 
(includes  deliverables). 

Per  ILSPR/CDRL. 

Full  program  with 
demonstrations. 

Full  program  including 
demonstrations. 

Full  to  maintenance 
concept. 

Manuals  per  CDRL. 

Full  maintenance 
training  to  the 
maintenance  concept 
with  2-year  turnover 
of  maintenance 
personnel . 

Contractor  field 
maintenance  pre- 
operational  training. 

Full  to  maintenance 
concept  with  deliver¬ 
ables. 

Full  exhibit  "C" 
effort  coincident 
with  LSA. 

Maintenance  concept  normally 
specified  by  contract. 


LSA  adjusted  to  reflect  contractor 
maintenance  -  no  cost  savings 
potential . 


Normally  no  savings  potential. 


Initial  maintenance  training  for 
contractor  personnel  reduced. 
Reduced  turnover. 


Normally  no  savings  potential. 


TABLE  1.  ALTERNATIVE  SUPPORT  CONCEPTS  -  COMPARATIVE  OVERVIEW  (Cont.) 


ELEMENT/FACTOR 

Spares 

Test  Equipment 

Manning 


ELEMENT/ FACTOR 


Type  of  contract 
Risks: 

-  Cost  Liability 


-  Transition 


Availability 
(Operational ) 

Cost  Control 


Cost  Visibility 


Incentives 


Configuration 

Control 

Maintenance 
Management 
and  Site 
Feedback 


Maintenance/Repair 

Concept 


Logistic  Support 
Analysis 

Reliability/ 

Maintainability 


CONVENTIONAL  SUPPORT  VS.  CONTRACT  MAINTENANCE  SERVICES  (Cont.) 

COMMERCIAL 

CONVENTIONAL  SUPPORT  COMPUTER  SUPPORT  _ COMMENTS _ 

Full  in  accordance  Contract  line  item;  Normally  no  savings  potential, 

with  maintenance  (max  funded)  based  on 

concept.  Gov't  support  method¬ 

ology. 

Standard  requirements  Contract  line  item.  Normally  no  savings  potential, 

in  accordance  with  (max  funded)  based  on 

maintenance  concept.  Gov't  support  method¬ 

ology. 

Variable  depending  Contract  line  item  Some  reductions  may  be  possible 

upon  availability  of  (max  funded)  based  on  depending  on  size  of  support 

personnel  and  skills.  Gov't  support  method-  effort. 

ology. _ _ _ 

CONVENTIONAL  SUPPORT  VS.  COMMERCIAL  COMPUTER  SUPPORT 
COMMERCIAL 

CONVENTIONAL  SUPPORT  COMPUTER  SUPPORT  _ COMMENTS _ 


MANAGEMENT  CONSIDERATIONS 


N/A 

Vendor  service. 

Can  be  service  agreement  ($/mo.) 
or  service  call  rate. 

Government  assumes 
responsibility. 

Normally  fixed  rate 
per  month  or  service 
call . 

Based  on  number  of  calls. 

Minimal  -  up  front 
acquisition. 

Government  assumes 
responsibility. 

Government  assumes 
risk. 

Dependent  on  vendor 
response  times. 

Vendor  manuals  are  normally 
included  as  part  of  device 
contract.  Include  option  for 
commercial  training. 

Government  assumes 
responsibility. 

Government  assumes 
responsibility. 

Government  assumes 
responsibility. 

Government  assumes 
responsibility. 

Conventional  contrac¬ 
tual  incentives. 

Conventional  contrac¬ 
tual  incentives. 

standard  military 
procedures 

Normal ly  assigned 
to  vendor. 

Updates  performed  by  vendor 
rep.  as  required. 

Standard  military 
procedures  -  (3M 

System) 

Vendor  service 
call  procedures. 

Government  may  transcribe  data 
to  Maintenance  Data  Collection 
System. 

LOGISTIC  SUPPORT  CONSIDERATIONS 

Per  maintenance 
analysis;  geared 
toward  Government 
support. 

Tailored  to  vendor 
service  guidelines. 

Heavy  use  of  computer  vendor 
factory  repair. 

Tailored  to  contract 
(includes  deliverables). 

Not  applicable  to 
computer. 

Proportional  savings  for  computer 
related  LSA  (as  applicable). 

Full  program  with 
demonstrations. 

Not  applied  to 
commercial  subsystems. 

Proportional  cost  savings. 

TABLE  1.  ALTERNATIVE  SUPPORT  CONCEPTS  -  COMPARATIVE  OVERVIEW  (Cont.) 


CONVENTIONAL  SUPPORT  VS.  COMMERCIAL  COMPUTER  SUPPORT  (Cont.) 


ELEMENT/FACTOR 

CONVENTIONAL  SUPPORT 

COMMERCIAL 

COMPUTER  SUPPORT 

COMMENTS 

Technical 

Manuals 

Full  to  maintenance 
concept. 

Vendor  manuals 
normally  supplied 
with  device. 

Minimal  cost  savings  opportunity. 

Training 

Full  maintenance 
training  to  the  main¬ 
tenance  concept  with 
2-year  turnover  of 
maintenance  personnel. 

Minimal  maintenance 
training  on  computer 
(operation  only). 

Significant  savings  since  computer 
maintenance  is  major  element  of 
training. 

Provisioning 

Full  to  maintenance 
concept  with  deliver¬ 
ables. 

None  -  use  of  vendor 
service  manuals. 

Proportional  savings  for  computer 
related  information. 

Spares 

Full  In  accordance 
with  maintenance 
concept. 

Use  of  vendor  procedures 
or  spare  whole  computer 
system. 

Savings  dependent  on  spares 
methodology  used. 

Test  Equipment 

Standard  requirements 
in  accordance  with 
maintenance  concept. 

Common  support  equipment 
only. 

Avoidance  of  special  test  equip¬ 
ment  for  computer/ peripherals. 

Manning 

Variable  depending 
upon  availability  of 
personnel  and  skills. 

Reduced  to  extent  com¬ 
puter  related. 

Proportional  savings  for  computer 
related  manning. 

CONVENTIONAL  SUPP6RT  VS. 

FIXED-PRICE  SUPPORT  WITHOUT  PENALTY/ INCENTIVE 

ELEMENT  FACTOR 

CONVENTIONAL  SUPPORT 

FIXED-PRICE  SUPPORT 
WITHOUT  PENALTY/ INCENTIVE 

COMMENTS 

MANAGEMENT  CONSIDERATIONS 

Type  of  contract 

Risks: 

-  Cost  Liability 

N/A 

Government  assumes 
responsibility. 

Fixed-price  (labor  and 
material). 

Limited  by  fixed-price 
amount. 

Contract  Includes  clauses  for  not 
to  exceed  price  quotes  on  2  option 
years.  Contract  value  renegoti¬ 
ated  annually. 

Contract  provides  Life  Cycle  Cost 
savings. 

-  Transition 

Availability 
(Operational ) 

Minimal  -  up  front 
acquisition. 

Government  assumes 
responsibility. 

Requires  transition 
package  option. 

Goal  only. 

Package  option  includes  conversion 
of  sufficient  minimal  data  for 
development  of  full  organic  docu¬ 
mentation  required. 

Cost  Control 

Government  assumes 
responsibility. 

Fixed-price  contract. 

Costs  can  change  with  annual 
renegotiation  of  support  contract 
(Inflation  effects  primarily). 

Cost  Visibility 

Government  assumes 
responsibility. 

ECP's  Induce  fixed- 
price  or  NTE  support 
Impact. 

ECP  costs  are  presented  as  fixed- 
price  changes  to  the  support  con¬ 
tract. 

Incentives 

Conventional  contrac¬ 
tual  Incentives. 

Same  as  conventional  and 
provision  for  R/M  product 
Improvements. 

Implied  incentives-assoclated  with 
fixed-price  contract. 
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TABLE  1.  ALTERNATIVE  SUPPORT  CONCEPTS  -  COMPARATIVE  OVERVIEW  (Cont.) 


CONVENTIONAL  SUPPORT  VS.  FIXED-PRICE  SUPPORT  WITHOUT  PENALTY/INCENTIVE  (Cont.) 


ELEMENT/FACTOR  CONVENTIONAL  SUPPORT 


FIXED-PRICE  SUPPORT 
WITHOUT  PENALTY/ INCENTIVE 


COMMENTS 


Configuration  Standard  military 

Control  procedures. 


Same  as  conventional  + 
provision  for  R/M  product 
improvements  and 
compatibility  changes. 


Contractor  has  additional  incen¬ 
tive  to  incorporate  R/M  data 
feedback  and  institute  product 
improvements. 


Maintenance 
Management 
and  Site 
Feedback 


Standard  military 
procedures  -  (3M 
System). 


Same  as  CMS  +  used  to 
initiate  product 
improvements. 


Product  improvements  are  implemen 
ted  because  of  incentive  to 
Improve  product  integrity  and 
availability.  All  changes  would 
indicate  support  impact  (if  any). 


LOGISTIC  SUPPORT  CONSIDERATIONS 


Maintenance/Repair  Per  maintenance 
Concept  analysis; geared 

toward  Government 
support. 


Tailored  to  contractor/ 
commercial  support 
methods. 


Logistic  Support 
Analysis 


Tailored  to  contract  Internal  analysis  only  Significant  savings  in  contrac- 
(includes  deliverables),  used  to  definitive  support  tual  value  of  Logistics  Support 
approaches/costs  keyed  to  Analysis  Program  potential 
Availability  goal.  savings  up  to  95%. 


Reliability/  Full  program  with  Sames  as  CMS. 

Maintainability  demonstrations. 


Technical 

Manuals 


Full  to  maintenance 
concept. 


Preliminaries  only  Potential  savings  of  50%  of 

(system  level)  +  corner-  Technical  Manuals  program, 
cial  documentation  and 
drawing  package. 


Training 


Provisioning 


Full  to  maintenance  May  require  fewer  per- 
concept.  sonnel  to  be  trained. 


Formal  course/training  data 
not  deliverable. 


Full  to  maintenance 
concept  with  deliver¬ 
ables. 


Limited/tailored  to 
contractor  style  parts 
list. 


Potential  savings  of  75%  of 
Provisioning  documentation  program. 


Spares 


Full  in  accordance 
with  maintenance 
concept. 


Selected  subsystems  may  Spares  may  be  procured  as  full 
be  purchased  as  complete  subsystems  providing  for  lot  size 
units  ("testbed"  approach),  economies. 


Test  Equipment  Standard  requirements  Test  bed  approach, 
in  accordance  with 
maintenance  concept. 


Test-bed  would  allow  for  off-line 
troubleshooting  utilizing  the 
test  bed  (less  test  equipment  to 
procure  and  maintain). 


Manning 


Variable  depending  May  require  fewer  support  Life  cycle  savings, 
upon  availability  of  personnel, 
personnel . 


CONVENTIONAL  SUPPORT  VS.  FIXED-PRICE,  PENALTY/ INCENTIVE  CONTRACTOR  SUPPORT 


FIXED-PRICE,  PENALTY/ 

INCENTIVE  CONTRACTOR 

ELEMENT/FACTOR  CONVENTIONAL  SUPPORT  SUPPORT  (FPP1CS) _ COMMENTS 


Type  of  contract  N/A 


Risks; 

-  Cost  Liability  Government  assumes 
responsibility. 


MANAGEMENT  CONSIDERATIONS 

Fixed-price  with  penalty/ 
Incentive  on  trainer 
availability. 


Limited  fixed-price 
amount. 


FPPICS  contract  Includes  clauses 
for  not  to  exceed  price  quotes  on 
2  option  years.  Contract  value 
renegotiated  annually. 

Contract  provides  Life  Cycle  Cost 
Savings. 
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TABLE  1.  ALTERNATIVE  SUPPORT  CONCEPTS  -  COMPARATIVE  OVERVIEW  (Cont.) 


CONVENTIONAL  SUPPORT  VS.  FIXED-PRICE,  PENALTY/INCENTIVE  CONTRACTOR  SUPPORT  (Cont.) 


ELEMENT/FACTOR 

CONVENTIONAL  SUPPORT 

FIXED-PRICE,  PENALTY/ 
INCENTIVE  CONTRACTOR 
SUPPORT  (FPPICS) 

COMMENTS 

-  Transition 

Availability 
(Operational ) 

Minimal  -  up  front 
acquisition. 

Government  assumes 
resonsibility. 

Requires  transition 
package  option. 

Guaranteed  contractual 
availability. 

Package  option  includes  conversion 
of  sufficient  minimal  data  for 
development  of  full  organic  docu¬ 
mentation  required. 

Cost  Control 

Government  assumes 
responsibility. 

Fixed-price  contract. 

Costs  can  change  with  annual 
renegotiation  of  support  contract 
(inflation  effects  primarily). 

Cost  Visibility 

Government  assumes 
responsibility. 

ECP's  include  fixed-price 
or  NTE  support  impact. 

Costs  are  presented  as  fixed-price 
changes  to  the  support  contract. 

Incentives 

Conventional  contrac¬ 
tual  incentives. 

Same  as  conventional  + 
provision  for  R/M 
product  improvements. 

Incentives  result  in  increased 
device  integrity  and  greater 
utilization  and  achivement  of 
training  mission. 

Configuration 

Control 

Standard  military 
procedures 

Same  as  conventional  + 
provision  for  R/M  product 
improvements  and  compati¬ 
bility  changes. 

Contractor  has  additional  incentive 
to  incorporate  R/M  data  feedback 
and  institute  product  improvements. 

Maintenance 
Management 
and  Site 

Feedback 

Standard  military 
procedures  -  (3M 
System). 

Same  as  CMS  +  used  to 
initiate  product 
improvements. 

Product  improvements  are  implemen¬ 
ted  because  of  incentive  to 
improve  product  integrity  and 
availability.  All  changes  would 

indicate  support  impact. 


LOGISTIC  SUPPORT  CONSIDERATIONS 

Maintenance/Repair  Per  maintenance  Tailored  to  contract/  Contractor  methods  geared  towards 


Concept 

analysis; geared 
toward  Government 
support. 

commercial  support 
methods . 

cost-effective  maximization  of 
availability. 

Logistic  Support 

Tailored  to  contract 
(includes  deliverables). 

Internal  analysis  only 
used  to  definitize  support 
approaches/costs. 

Significant  savings  in  contractual 
value  of  Logistics  Support  Analysis 
Program  potential  savings  up  to  95%. 

Reliability/ 

Maintainability 

Full  program  with 
demonstrations. 

Demo's  not  required 
because  of  penalty/ incen¬ 
tive,  fixed-price. 

Potential  savings  of  approximately 
25%  of  contractual  R/M  program. 

Technical 

Manuals 

Full  to  maintenance 
concept. 

Preliminaries  only  (sys¬ 
tem  level)  +  comnercial 
documentation  and  drawing 
package. 

Potential  savings  of  50%  of  Tech¬ 
nical  Manuals  program. 

Training 

Full  maintenance 
training  to  the  main¬ 
tenance  concept  with 
2-year  turnover  of 
maintenance  personnel. 

Same  as  CMS;  may  require 
fewer  personnel  to  be 
trained. 

Potential  savings  measured  over 
system  life  cycle. 

Provisioning 

Full  to  maintenance 
concept  with  deliver¬ 
ables. 

Limited/tailored  to  con¬ 
tractor  style  parts  list. 

Potential  savings  of  75%  of 
Provisioning  documentation  program. 

Spares 

Full  In  accordance 
with  maintenance 
concept. 

Selected  subsystems  may  Spares  may  be  procured  as  full  sub- 

be  purchased  as  completed  systems  providing  for  lot  size 
units  ("test  bed"  approach),  economies. 
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TABLE  1.  ALTERNATIVE  SUPPORT  CONCEPTS  -  COMPARATIVE  OVERVIEW  (Cont.) 


CONVENTIONAL  SUPPORT  VS.  FIXED-PRICE  SUPPORT  WITHOUT  PENALTY/INCENTIVE  (Cont.) 


ELEMENT/FACTOR  CONVENTIONAL  SUPPORT 


FIXED-PRICE.  PENALTY/ 
INCENTIVE  CONTRACTOR 
SUPPORT  (FPPICS) 


COMMENTS 


Test  Equipment  Standard  requirements  Test  bed  approach, 
in  accordance  with 
maintenance  concept. 


Test-bed  would  allow  for  off-line 
troubleshooting  utilizing  the 
test  bed  (less  test  equipment  to 
procure  and  maintain). 


Manning 


Variable  depending 
upon  availability  of 
personnel  and  skills. 


May  require  fewer  support  Life  cycle  savings  through  FPPICS 
personnel  -  fixed  level  approach.  Also,  availability  not 
established  by  contract.  jeopardized  by  manning  variations. 


stocking  the  supply  system  with  parts.  A  cost- 
effective  option  is  to  access  the  support  system 
established  by  the  computer  manufacturer. 

This  approach  offers  cost  savings  in  training, 
documentation,  parts  support  and  on-site  mainten¬ 
ance  personnel.  The  approach  also  has  some  risks 
which  must  be  addressed: 


•  On-call  computer  services  may  not  provide 
the  desired  availability  of  the  training 
device  in  the  event  of  computer  malfunc¬ 
tions. 

•  Problems  which  involve  the  interface 
between  the  cormtercial  computer  and  the 
training  device  may  be  more  difficult  to 
solve. 

•  Careful  configuration  management  and 
control  of  commercial  revision  activity 
must  be  exercised  by  Government  main¬ 
tenance  personnel.  This  is  necessary 
to  insure  that  revisions  which  may  be 
incorporated  by  the  computer  manufac¬ 
turer  during  maintenance  (on-site  or 
factory  repairs)  do  not  introduce 
system  (hardware  or  software)  malfunc¬ 
tions. 

•  Logistics  management  personnel  must 
maintain  a  close,  on-going  awareness 
of  commercial  repair  capabilities  to 
avoid  excessive  dependence  on  a  single 
source  of  support.  (Many  commercial 
repair/ support  agreements  carry  a 
stipulation  that  the  manufacturer 
reserves  the  right  to  discontinue  sup¬ 
port  with  minimum  notice.) 

Fixed-Price  Support  Without  Penalty/ Incentive 

This  method  offers  an  implied  warrantee  If 
material  costs  are  Included  In  the  fixed-price 
amount  since  lower  equipment  reliability  would 
cause  Increased  material  expenses  without  reim¬ 
bursement.  This  approach  also  offers  potential 
cost  savings  In  many  areas  of  the  acquisition 
contract  but  does  not  offer  assurance  of  a 
required  level  of  operational  performance  or 
availability. 


Penalty/Incentive  Support 

The  introduction  of  the  penalty/ incentive 
feature  to  contractor  support  is  the  primary 
focus  of  this  paper.  Since  the  application  of 
the  penalty/incentive  clause  is  based  on  trainer 
operational  availability, a  substantial  confi¬ 
dence  level  in  meeting  operational  training  needs 
is  achieved.  As  a  result,  many  of  the  conven¬ 
tional  controls,  reviews,  analyses,  conferences, 
reports,  data,  tests,  and  demonstrations  can  be 
reduced  or  removed  from  program  requirements 
while  at  the  same  time,  taking  advantage  of  the 
basic  cost  savings  associated  with  the  applica¬ 
tion  of  contractor  support.  As  shown  in  Table  1, 
cost  savings  and  cost  avoidance  associated  with 
this  concept  are  available  in  almost  every 
element  of  logistic  support. 

Penalty/Incentive  Formula 

The  establishment  of  a  penalty/incentive 
formula  must  be  tailored  to  the  specific  program 
under  consideration.  The  formula  should  be  based 
upon  mutual  understanding  of  the  technical  and 
costs  risks  associated  with  the  equipment  perform¬ 
ance  and  support  effort.  In  general,  if  the 
operating/support  environment  is  well  defined  as 
described  in  CONCLUSIONS/SUMMARY  and  the  contrac¬ 
tor  has  high  confidence  in  the  system  Reliability 
and  Maintainability  characteristics  (through  ^iald 
experience  or  internal  tests  for  example),  then 
the  availability  targets  can  be  set  at  a  compara¬ 
tively  high  level  (in  the  95X  to  98%  range).  It 
is  important  that  the  contractor  provide  feedback 
to  the  customer  during  contractor  support  planning 
as  to  the  Incremental  costs  associated  with  selec¬ 
ted  levels  of  availability  using  the  contractor's 
planned  support  methods.  Further,  the  Government 
should  consider  the  operating/training  environment 
(as  described  in  CONCLUSIONS/SUMMARY)  in  determin¬ 
ing  availability  requirements  and  chargeabi 1 1 ty  of 
downtime  described  below. 

KEY  ELEMENT:  D  CONSIDERATIONS 

This  section  sunrn,  izes  the  key  elements  and 
considerations  asso<  ited  with  the  application  of 
a  fixed-price,  penal  entive  contractor  sup¬ 
port  approach  with  p.  „ /incentive  based  on 
trainer  operational  availability.  It  is  essential 
that  the  concept  be  considered,  planned  and 
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defined  at  the  earliest  stage  of  the  contract  In 
order  for  cost  savings  and  avoidance  to  be  real¬ 
ized  and  to  minimize  problems  In  execution  of  the 
concept. 

Program  Timing  and  Criteria 

The  most  effective  application  of  fixed-price, 
penalty/incentive  costs  savings  is  on  new  programs. 
This  is  evident  from  Table  1,  which  indicates  that 
many  of  the  cost  savings  are  derived  from  the  log¬ 
istics  element  requirements  in  the  initial  equip¬ 
ment  contract.  The  ideal  fixed-price,  penalty/ 
incentive,  contract  support  program  is  one  in 
which  several  trainers  will  be  installed  in  one 
facility  to  achieve  maximum  manning  productivity. 
Small,  deployable,  disbursed,  mobile  trainers,  on 
the  other  hand,  are  not  good  candidates.  In 
addition,  the  existing  programs  being  currently 
supported  by  the  Government  or  a  contractor  are 
not  good  candidates  since  the  Government's  sup¬ 
port  capability  has  already  been  acquired  and  the 
costs  are  "sunk".  However,  consideration  can  be 
given  to  converting  existing  military  training 
equipment  support  facilities  to  contractor  sup¬ 
port  in  cases  where  the  trainer  is  a  large  one- 
of-a-kind,  highly-complex  unit,  or  if  sources  of 
repair  parts  through  standard  Government  channel 
are  diminishing.  In  these  cases,  contractor 
support,  however,  does  not  offer  dramatic  cost 
savings/avoidance  but  may  offer  better  mainten¬ 
ance  personnel  stability/skill  and  more  flexib¬ 
ility  of  support/repair  approaches.  It  is 
important  to  recognize  in  considering  converting 
an  existing  military  support  facility  to  contrac¬ 
tor  support  that  there  is  likely  to  be  an  under¬ 
standable  resistance  to  change  on  the  part  of 
the  existing  support  personnel  which  may  outweigh 
the  less  than  dramatic  cost  advantages  of 
converting  to  contractor  support  for  a  fielded 
system.  It  is  not  the  intent  of  this  paper  to 
discuss  the  potentially  controversial  pros  and 
cons  associated  with  military  versus  contractor 
support. 

Changeability  of  Downtime 

Since  the  key  to  the  penalty/ incentive 
approach  is  in  the  method  of  calculating  the - 
penalty/incentive,  it  must  be  specific  and 
clearly  defined  in  contractual  terms  for  calcu¬ 
lating  chargeability  of  trainer  downtime.  The 
most  manageable  definition  of  downtime  is  as 
defined  by  MIL-STD-23991:  When  the  trainer  is 
required  for  scheduled  student  training  and^  is 
not  available  for  that  training  as  a  direct 
result  of  a  system  malfunction.  Downtime  Is  not 
chargeable  under  the  circumstances  where  the 
trainer  is  not  needed  for  student  training  for 
specific  limited  periods;  or  when  an  apparent 
malfunction  is  determined  to  be  the  result  of 
improper  equipment  operation;  or  when  a  failure 
or  malfunction  is  the  result  of  conditions  out¬ 
side  of  the  contractor's  control  such  as  facility 
problems,  student  misuse,  etc. 

On-site  Flexibility 

While  there  must  be  clear  contractual 
definitions,  It  is  Important  that  these  defini¬ 
tions  do  not  excessively  constrain  on-site 
personnel  In  adjusting  to  variations  in  the 
training  operational  environment.  Further,  If 


disagreements  occur  as  to  downtime  chargeability 
(which  experience  has  shown  has  been  very  minimal 
to  date),  these  disagreements  are  best  handled 
by  referring  them  through  an  off -site  administra¬ 
tive  channel  for  resolution.  Figure  1  is  an 
example  of  the  form  used  for  recording  downtime 
and  the  conditions  at  the  time  of  the  occurrence. 
This  form  has  been  in  use  on  the  existing  pen¬ 
alty/incentive  T-34C  CPT  contract. 
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Figure  1.  Lost  Training  Time  Record 


Maintenance  Personnel  Flexibility.  In 
general,  specialization  of  maintenance  personnel 
Is  not  desirable  from  a  planning  viewpoint.  Main¬ 
tenance  and  support  personnel  should  not  be  hired 
or  placed  on  a  crew  with  known  skill  limitations 
(e.g..  Computer  Specialist;  Inventory  Specialist). 
Gould/SSD  experience  indicates  that  qualified 
individuals  will  come  equipped  with  some  special 
skills  from  prior  assignments;  and  that  some 
specialization  is  a  natural  development  as  support 
and  maintenance  activities  proceed.  In  addition, 
at  least  one  of  the  senior  members  of  the  on-site 
support  team  should  be  designated  as  the  training 
focal  point  to  Insure  an  on  going,  optimum  mix  of 
skills  as  maintenance  needs  dictate. 

Government  Maintenance  Data  Utilization 


Use  of  on-site  Government  maintenance  data 
capabilities  Is  highly  desirable  whenever  possible. 
Best  advantage  is  taken  when  the  contractor  sup¬ 
port  team  develops  a  workable  Work  Unit  Code 
structure  for  the  particular  device/equipment 
being  supported  and  develops  unique  applications 
of  the  local  use  blocks  of  the  maintenance  data 
collection  form  as  required.  In  this  way,  the 
Maintenance  Summary  Reports  which  are  developed 
from  data  collection  system  can  be  used  as  both 
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deliverable  reports  to  the  Government  (an  addi¬ 
tional  cost  avoidance)  and  can  also  be  returned 
to  the  factory  for  maintenance  and  support 
management  purposes. 

Additional  Trainer  Hours 

Additional  trainer  hours  must  be  accounted 
for  in  the  support  contract  in  the  event  that 
weather,  student  progress,  student  load,  or  other 
variations  in  training  workload  require  addi¬ 
tional  hours.  Additional  trainer  hours  are  norm¬ 
ally  provided  or  performed  during  weekends  or  if 
it  is  critical,  during  the  maintenance  shift. 

Factory  Support 

Factory  support  is  a  normal  part  of  contrac¬ 
tor  support.  It  includes  overall  support  manage¬ 
ment,  budgeting,  cost  reporting,  change  review 
and  ECP  preparation,  reliability  and  maintain¬ 
ability  analyses,  vendor  or  factory  repair 
management,  etc.  This  insures  that  the  on-site 
effort  proceeds  in  the  general  direction  which 
was  originally  envisioned  by  both  the  customer 
and  the  supporting  contractor. 

Preoperational  Support 

This  is  a  particularly  important  element. 

It  is  unlikely  that  a  given  contractor  will  have 
a  fully  qualified  staff  of  maintenance  techni¬ 
cians  automatically  available  as  a  result  of  a 
development/production  effort.  The  development 
of  this  maintenance  staff  must  be  a  clear-cut 
element  of  the  contractor  support  option  and  must 
be  clearly  identified  as  a  support  cost  for  the 
program  prior  to  the  device  installation/selloff. 

Change  Management 

Changes  are  inevitable  during  any  equipment 
support  effort.  Again,  the  key  is  planning  to 
accommodate  change.  As  shown  in  Table  1,  an 
important  advantage  to  fixed-price,  penalty/ 
incentive  contractor  support  is  the  cost  visi¬ 
bility  and  control  when  changes  are  appropriate. 
Since  under  fixed-price,  penalty/incentive  con¬ 
tractor  support,  the  contractor  is  reponsible 
for  development  and  implementation  of  a  change, 
the  support  impact  costs  are  identified  and 
included  in  the  ECP  not  as  speculation  or  pre¬ 
diction  but  as  proposed  costs.  This  insures 
management,  budgeting,  timing,  and  equipment 
utilization  plans  are  all  addressed  and  accoun¬ 
ted  for. 

The  primary  sources  of  change  are: 

Commercial  Equipment.  Product  improvements 
(model  changes,  updates,  fixes,  etc.). 

ReHabllity/Mainta inability  (R/M).  Changes 
identified  through  first-hand  trainer operation/ 
utilization  (some  of  these  may  be  submitted  at  no 
cost  since  the  contractor  would  normally  propose 
such  R/M  changes  based  upon  a  payback  analysis). 

Compatibility  Changes.  Changes  to  keep  the 
trainer  at  the  same  operational /performance  level 
as  the  operational  equipment  being  simulated. 
Compatibility  changes  are  the  most  common  for 
trainers  and  contract  provisions  for  them  should 


be  specified  in  the  contract  to  tie  the  Govern¬ 
ment,  operational  equipment  manufacturer,  and 
training  equipment  supplier  into  the  same 
configuration  management  process. 

TRANSITION  PACKAGE 

In  planning  for  any  form  of  contractor 
support,  it  would  be  judicious  for  the  Govern¬ 
ment  to  address  the  possibility  of  a  change  in 
support  responsibility  at  some  point  in  the 
Operation  and  Support  (O&S)  phase  of  a  program. 
Even  though  such  changes  would  be  unlikely,  they 
should  be  considered  to  avoid  the  negative  support 
impact  of  contractor's  not  bidding  for  support 
contracts,  unexpected  or  unjustified  cost 
increases,  or  other  major  changes  in  support 
philosophies. 

Therefore,  Gould/SSD  has  developed  the 
"Transition  Package"  concept  under  which  the 
contractor  includes  a  fixed-price  option  to  allow 
a  smooth  turnover  from  one  contractor  to  another, 
or  to  Government  personnel.  This  option  should 
include  the  minimum  necessary  logistic  elements 
to  insure  a  smooth  transition.  These  elements 
would  include  as  a  minimum: 

•  Maintenance  drawings. 

•  Provisioning  documentation. 

•  Software  documentation. 

•  Technical  manuals. 

•  Special  purpose  tools  and  test  equipment. 

•  On-the-job-training. 

As  part  of  the  contract,  a  transition  plan 
should  be  included  in  the  basic  contract  as  a 
deliverable  item  to  the  Government,  to  be  updated 
on  a  regular  basis  with  sufficient  scope  and  depth 
to  insure  an  orderly  transition  should  the  need 
occur.  The  format  would  be  similar  to  an  Integra¬ 
ted  Logistics  Support  Plan. 

A  penalty  clause  should  be  included  in  the 
contract  to  preclude  the  contractor  from  placing 
the  Government  in  an  untenable  sole  source  situ¬ 
ation,  in  that  the  contractor  must  provide  the 
Government  sufficient  time  to  transition  the 
device  support  from  one  contractor  to  another,  or 
to  Government  personnel  as  the  case  may  be. 

The  specific  contents  of  the  transition 
package  should  be  discussed  and  agreed  to  at  the 
same  time  as  contractor  support  (or  a  contractor 
support  option)  is  exercised.  This  timing  is 
essential  to  insure  that  proper  mix  between  the 
requirements  of  the  contractor  support  contract 
and  the  necessary  (but  not  duplicative)  elements 
of  the  Transition  Package. 

COST  PROFILES  AND  COMPARISONS 

Figure  2  indicates  typical  cost  profiles  for 
the  alternatives  discussed  in  this  paper.  The 
cost  profiles  shown  are  a  summary  level  consoli¬ 
dation  of  cost  information  derived  from  the  pre¬ 
viously  mentioned  programs  and  the  estimated 
savings  associated  with  FPPICS.  Since  each 
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Figure  2.  Comparative  Cost  Profiles  for  Contractor  Support  Alternatives 

(when  applied  at  program  initiation  for  a  typical  simulator  program). 


program  contains  its  own  unique  requirements,  the 
profiles  have  been  averaged  so  that  they  reflect 
a  "typical"  trainer  program  and  are  representative 
of  the  cost  differences  at  the  system  level.  Of 
course,  any  such  analysis  of  this  type  is  inher¬ 
ently  subject  to  analytical  judgment  and  to 
variations  in  implementation.  The  fixed-price, 
penalty/incentive  cost  savings  approach  achieves 
the  maximum  support  savings  when  applied  at 
program  initiation.  The  extent  of  savings  is 
dependent  primarily  on  both  what  the  conventional 
support  requirements  would  start  with  (i.e.,  LSA 
or  Functionally  Oriented  Technical  Manuals  would 
increase  the  conventional  support  basic  costs), 
as  well  as  the  cost  assumptions  used  for  salary/ 
overhead  rates  for  Government  maintenance  per¬ 
sonnel  (which  may  vary  according  to  the  extent 
of  Government  overhead  which  is  allocable  to  the 
Government  support  direct  costs). 

CONCLUSIONS/SUMMARY 

Based  on  the  discussion  above,  fixed-price, 
penalty/ incentive  contractor  support  offers 
significant  advantages  in  terms  of  cost  savings/ 
avoidance,  availability  and  proficiency  of  sup¬ 
port  personnel,  and  "Insured"  equipment  perfor¬ 
mance  for  training.  To  pursue  this  approach  in 
general  and/or  for  a  particular  contract,  the 
following  recommendations  are  made: 

Description  of  the  Qperatinq/Support  Environment 

It  is  important  that  the  customer  describe 
specifically  the  intended  equipment  utilization 


methods,  utilization  rates,  operating  hours, 
student  flow,  team  training  versus  individual 
training,  the  role  of  the  instructor/operator  in 
system  maintenance  and  any  other  application 
unique  information. 

Development  of  Model  Contracts 

The  Government  Procurement  Agencies  should 
develop  a  "standard"  for  contractor  support 
options/contracts  including  planning  checklists, 
standard  contract  provisions,  and  clauses,  etc., 
covering  the  issues  in  this  paper  and  any  other 
applicable  areas  so  that  the  management  learning 
curve  is  realized  in  the  tailored  application  of 
contractor  support  approaches.  The  "standard" 
should  be  developed  jointly  between  military 
training  procurement  agencies  and  industry 
by  preparation  of  initial  drafts,  conferences, 
interim  guidel ines,  and,  where  appropriate,  trial 
appl ications. 

Similarities  Between  Penalty/Incentive  Support 
and  Existing  Comriercial  Equipment  Support 

It  should  be  noted  that  most  commercial 
equipment  suppliers  (e.g.,  commercial  computer 
manufacturers)  offer  contractual  support  (or 
"customer  service")  as  a  matter  of  routine.  And 
as  described  above,  commercial  support  of  compu¬ 
ters  is  being  considered  more  and  more  for  train¬ 
ing  equipment  computers  when  the  appropriate 
conditions  exist.  Since  many  military  simulators 
are  used  in  a  predicatble,  fixed-facility  or 
institutional  application  and  since  they  make  use 
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ABSTRACT 

This  paper  suggests  that  contractor  maintenance  training  acquisition 
procedures  used  by  Government  are  often  not  cost  effective  and  suggests  a 
paradigm  for  the  development  of  acquisition  procedures  which  will  increase 
training  effectiveness  and  reduce  costs.  Government  often  levies  excessive 
and  unnecessary  restrictions  on  contractor  by  insisting  that  certain  military 
standards  and  data  item  descriptions  be  followed  even  in  those  circumstances 
when  such  standards  are  both  inappropriate  and  undesi reable.  This  approach 
results  in  increased  training  acquisition  costs. 


INTRODUCTION 

Contractor  maintenance  training 
courses  for  newly  procured  training  de¬ 
vices  are  acquired  for  one  reason  only:  to 
ensure  that  qualified  personnel  will  be 
available  to  maintain  the  device  through¬ 
out  its  life  cycle.  These  courses  must 
achieve  an  appropriate  balance  of  various 
factors  such  as  device  maintenance  concepts 
performance  objectives  to  be  attained,  and 
availability  of  training  funds.  Ideally, 
a  course  graduate  will  have  the  ability  to 
effectively  diagnose  and  isolate  faults  and 
repair  the  device  within  the  specified  mean 
downtime  limits.  The  training  acquisition 
program  also  procures  training  documenta¬ 
tion  and  curricula  to  be  used  for  retrain¬ 
ing  in  the  out  years. 

In  the  past  few  years,  major  forces 
have  impacted  heavily  on  the  training 
device  world.  The  scarcity  and  cost  of 
fuel  have  caused  an  increased  demand  for 
training  devices  to  re place  training 
normally  conducted  in  operational  situa¬ 
tions.  To  meet  this  need,  more  complex 
and  sophisticated  training  devices  are 
being  developed.  As  technology  advances, 
maintenance  concepts  change  in  an  attempt 
to  keep  pace.  Concurrently,  it  has  become 
apparent  that  the  input  skill  level  of 
trainees  is  continually  declining.  Train¬ 
ing  funds  are  always  in  the  vanguard  of 
budget  cuts.  Given  these  factors  as  more 
or  less  permanent  fixtures  for  the  fore¬ 
seeable  future,  it  is  imperative  that 
cost  effective,  dynamic  maintenance  train¬ 
ing  acquisition  concepts  be  developed  if 
the  training  device  community  is  to  remain 
responsive  to  its  overall  mission. 

PROBLEM  ANALYSIS 

Evidence  which  suggests  that  current 
maintenance  training  acquisition  procedures 
are  not  cost  effective  can  be  traced  to 
one  major  underlying  factor:  excessive 
Government  regulation  of  industry.  Several 


high  level  directives  have  been  issued  in 
an  attempt  to  deal  with  this  problem,  four 
of  which  are  expressly  germane  to  the  main¬ 
tenance  training  acquisition  process .  The 
Defense  Acquisition  Regulations  (DAR)  set 
forth  the  basic  legal  doctrine defining 
the  role  of  Government  and  contractor  re¬ 
lationships.  (1)  Office  of  Management  and 
Budget  (0MB)  Circular  A-109  Major  Systems 
Acouisi tions  further  defines  these  roles 
and  expl icitly  recommends  that  Government 
establish  system  performance  objectives 
and  allow  the  contractor  to  develop  de¬ 
tailed  design  and  training  objectives.  (7) 
The  policies  of  0MB  Circular  A-109  are 
further  explicated  in  0MB  Curcular  A-76, 
Policies  for  Acquiring  Commercial  or  In¬ 
dus  tri  al  Products  and  Services  for  Govern- 


struction  (SECNAVINST)  4200. 27A  Proper 
Use  of  Contractor  Personnel  cl  early  de¬ 
fines  the  proper  role  of  contractor  per¬ 
sonnel.  (9) 

In  spite  of  these  and  other  policy 
documents  and  legal  prohibitions,  current 
acquisition  procedures  continue  to  regulate 
almost  every  step  of  a  contractor’s  main¬ 
tenance  training  development  efforts.  In 
the  initial  phases  of  procurement,  detailed 
overly  specific  Requests  For  Proposal  (RFP) 
limit  the  creativity  and  resourcefulness 
of  the  contractor.  (10)  Once  the  contract 
has  been  awarded,  the  administration  of 
the  maintenance  training  acquisition  ele¬ 
ment  often  veers  away  from  the  spirit  and 
the  letter  of  the  directives  cited  above. 
Both  of  these  situations  are  results  of 


the  current  maintenance  training  acquisi¬ 
tion  philosophy,  which  has  evolved  into  a 
rigidly  structured  process  of  constant 
control  and  monitoring  of  the  contractor's 
efforts  from  program  conception  to  contract 
conclusion.  Government  personnel  from  the 
functional  areas  of  engineering,  management, 
logistics,  contracting,  and  training  con¬ 
tinually  tell  the  contractor  what  and  how 
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much  training  is  required,  how  to  develop 
the  training  curriculum,  how  to  write 
curriculum  data  items,  and  what  instruc¬ 
tional  procedures  will  be  followed.  Par¬ 
ticularly  in  the  curriculum  development 
process  the  contractor  is  bound  to  pro¬ 
cedures  outlined  and  delineated  in  a 
plethora  of  regulations  and  instructions. 
The  entire  training  course  and  curriculum 
development  process  is  tied  to  the  contents 
of  the  Military  Standard  governing  con¬ 
tract  training  programs  (MIL-STD  1379A) 
and  various  Data  Item  Descriptions  (DIDs). 
(2)  These  standards  specify  that  the 
format,  content  areas,  and  virtually  all 
other  facets  of  the  curriculum  associated 
documents  will  be  developed  in  accordance 
with  highly  structured  curriculum  develop¬ 
ment  models. 

However,  highly  structured  models  are 
not  panaceas  for  all  curriculum  develop¬ 
ment  efforts  and  problems.  While  an 
instructional  system  development  process 
provides  excellent  curriculum  development 
guidelines  and  procedures  for  efforts  on 
training  courses  of  a  continuing  or  "pipe¬ 
line"  nature,  this  process  is  not  always 
appropriate  for  developing  curricula  for 
what  are  essentially  one  time  factory 
training  courses.  Curriculum  develop¬ 
ment  utilizing  rigidly  structured  pro¬ 
cedures  and  models  requires  a  substantial 
expenditure  of  time,  effort,  and  money. 

Such  expenditures  can  be  justified  for  a 
"pipeline"  type  training  course;  they 
cannot  be  justified  for  a  course  that  will 
be  presented  only  one  or  two  times  during 
a  device  life  cycle  even  if  quality  prod¬ 
ucts  are  received. 

Unfortunately,  quality  products  are 
rare.  A  large  number  of  curricula  are 
accepted  by  Government  because  they  meet 
all  the  legal  requirements  of  the  con¬ 
tract,  not  because  they  are  of  high 
quality.  While  in  the  process  of  pre¬ 
paring  and  responding  to  RFPs,  neither 
Government  nor  industry  pays  proper 
attention  to  the  difficult  nature  of 
applying  the  instructional  systems  de¬ 
velopment  process  specified  by  MIL-STD 
1379A.  As  a  result,  contractors  often 
misjudge  the  effort  required  to  meet 
standards  specified  by  MIL-STD  1379A  and 
associated  DIDs.  A  recent  study  has 
partially  documented  some  of  the  curricula 
development  problems  encountered  by  a  pro¬ 
fessional  training  development  firm  using 
these  Government  mandated  procedures.  (3) 
These  problems  and  others  can  reasonably 
be  expected  to  pose  major  concerns  for 
hardware  oriented  contractors,  particu¬ 
larly  those  who  rarely  employ  personnel 
with  instructional  systems  development 
experience.  Contractors  usually  appoint 
engineers,  technical  writers,  publication 
specialists,  or  personnel  from  a  public 
school  or  university  background  to  head 
and  staff  their  training  departments. 

The  acceptance  of  mediocre  and  mar¬ 
ginal  quality  curricula  is  also  influenced 
greatly  by  the  delivery  schedule  of  curri¬ 
cula  data  items.  These  items  are  normally 
delivered  toward  the  end  of  the  acquisi¬ 
tion  cycle.  Shrewd  contractors  realize 


that  at  this  stage  of  the  cycle,  hardware 
oriented  Government  program  managers  are 
growing  weary  of  problems,  especially  if 
device  delivery  has  been  delayed  one  or 
more  times  and  cost  overruns  have  occurred; 
they  are  anxious  to  close  the  contract. 

All  too  often,  various  pressures  cause 
curricula  data  item  discrepancies,  which 
threaten  further  delays,  to  be  inadver¬ 
tantly  overlooked. 

Over  the  years,  various  management 
policies  have  evolved  which  seldom  allow 
training  acquisition  personnel  to  simply 
reject  an  unsatisfactory  product;  a  de¬ 
tailed  justification  explaining  why  the 
product  was  rejected  must  be  provided. 
Astute  contractors  are  aware  that  if  these 
detailed  justifications  are  not  provided, 
low  quality  curricula  data  items  can  be 
produced  at  a  sizable  cost  savings.  Once 
detailed  comments  are  provided,  all  the 
contractor  need  do  is  incorporate  these 
comments  into  his  own  format  and  resubmit. 
This  editing  function  constitutes  a  very 
time  consuming  process  for  training  ac¬ 
quisition  personnel.  Situations  thus  tend 
to  develop  wherein  Government  personnel 
provide  detailed  instruction  in  curriculum 
development  prior  to  the  delivery  of  final 
data  items  simply  to  preclude  becoming  a 
contractor's  proofreader  and  editor. 

As  a  re  suit,  in  many  instances, 
training  acquisition  personnel  perform  a 
significant  portion  of  the  contractor's 
work  in  order  to  receive  an  acceptable, 
quality  product.  This  process  keeps  these 
personnel  from  performing  more  important 
functions,  such  as  maintenance  training 
needs  analysis,  development  of  perform¬ 
ance  objectives,  and  training  evaluation. 

It  is  also  in  direct  violation  of  various 
policies  and  directives. 

Still  another  problem  inherent  within 
the  current  training  acquisition  philosophy 
is  the  procurement  of  unnecessary  curri¬ 
culum  data  items.  The  typical  curriculum 
acquired  for  a  maintenance  training  course 
consists  of  a  Training  Course  and  Curri¬ 
culum  Outline,  an  Instructor's  Guide,  a 
Student  Guide,  Audio  Visual  Aids,  Tests, 
Evaluation  Forms,  and  an  On-The-Job  Train¬ 
ing  Handbook.  The  average  cost  for  these 
data  items  is  $64,000.  with  an  upward 
range  of  well  over  $250,000.  Yet  it  is 
seldom  questioned  whether  all  of  these 
items  are  actually  required  to  achieve  an 
acceptable  maintenance  training  course. 

Curriculum  data  items  produced  for  a 
maintenance  training  course  consist  of 
maintenance  documentation  packaged  in  an 
instructional  systems  format.  This  same 
documentation  is  also  delivered  under 
another  contract  line  item.  Since  both 
line  items  are  priced  out  as  original 
development  effort,  in  effect  Government 
is  paying  twice  for  one  product,  namely 
technical  maintenance  documentation . 

As  an  example,  an  On-The-Job  Training 
Handbook  is  little  more  than  a  guide  and  a 
supplement  to  device  maintenance  documen¬ 
tation.  Still,  Government  pays  up  to 
$65,000.  and  over  to  acquire  On-The-Job 
Training  Handbooks.  Perhaps  this  ex¬ 
penditure  might  be  justified  if  Instructor 


483 


Guides,  Student  Guides,  Audio  Visual  Aids, 
and  Text  materials  were  not  also  acquired. 
Some  Government  agencies  are  beginning  to 
deal  with  overprocurement  problems  such 
as  these  by  establishing  Maximum  Data 
Lists  and  Data  Review  Boards.  Although 
such  tactics  produce  limited  results, 
they  are  in  effect  well  meaning  attempts 
to  "fix"  a  system  that  appears  to  require 
a  major  overhaul.  There  is  no  rational 
justification  for  procurement  of  all  these 
curriculum  items  in  addition  to  the  re¬ 
quired  maintenance  documentation. 

Further,  contracts  normally  require 
that  the  person  designated  as  the  main¬ 
tenance  training  course  instructor  be  a 
member  of  the  team  that  develops  main¬ 
tenance  and  other  device  documentation. 
During  this  process,  the  instructor 
necessarily  has  to  compile  notes,  diagrams, 
and  other  materials  that  will  be  used  in 
teaching  the  course.  Prior  to  the  appli¬ 
cation  of  an  instructional  systems  de¬ 
velopment  process  to  almost  every  curri¬ 
culum  development  situation,  effective 
maintenance  training  courses  were  con¬ 
ducted  using  the  instructor's  personally 
prepared  lesson  plans,  professional  know¬ 
ledge  and  expertise,  and  the  maintenance 
documentation . 

The  only  logical  rationale  for  paying 
the  contractor  to  restructure  all  of  this 
technical  knowledge  and  information  into 
a  specific  instructional  systems  format 
appears  to  be  that  the  curriculum  data 
items  will  be  required  for  retraining 
efforts  in  the  out  years.  However,  this 
rationale  does  not  hold  up  under  close 
scrutiny.  By  the  time  retraining  is  re¬ 
quired  either  device  modifications  have 
made  the  training  material  obsolete  or 
the  material  has  been  lost.  Although 
one  or  two  curriculum  data  items  may  be 
used  in  retraining,  the  entire  curriculum 
package  will  be  used  only  once  or  twice 
in  a  fifteen  year  period,  if  used  at  all. 
More  often  than  not,  situational  re¬ 
quirements  dictate  that  personnel  who 
conduct  retraining  develop  their  own 
materials  and  methods  to  meet  current 
needs.  In  essence,  a  qualified  instructor 
and  a  well  developed  and  up  to  date  main¬ 
tenance  documentation  package  are  the 
only  elements  required  to  conduct  formal 
retraining  courses.  Moreover,  individual¬ 
ized  learning  programs  such  as  Technical 
Hands  On  Training  (THOT)  packages  are 
finding  greater  acceptance  and  appli¬ 
cation  in  the  retraining  effort.  Since 
Government  personnel  have  the  ability  to 
develop  THOT  type  programs  from  high 
quality  and  complete  maintenance  docu¬ 
mentation,  the  requirement  to  procure 
fully  developed,  rigidly  formatted 
curriculum  is  obviated. 

Another  dubious  cost  factor  can  be 
found  under  the  heading  "Instructor 
Preparation  Time."  As  noted  earlier, 
the  instructor  is  normally  required 
contractually  to  be  a  member  of  the  team 
which  develops  maintenance  documentation. 
Still,  contractors  insist  upon  an  in¬ 


structor  preparation  time  within  a  ratio 
of  2  to  6  times  course  length.  This  means 
that  if  a  maintenance  course  is  scheduled 
to  be  6  weeks  long  (240  hours),  a  con¬ 
tractor  may  submit  charges  for  preparation 
time  that  range  from  480  hours  up  to  1440 
hours.  At  an  average  conservative  rate  of 
$35.  an  hour  cost  to  Government,  this 
translates  into  a  price  range  of  $16,800. 
to  $50,400.  for  preparation  time.  Since 
the  instructor  is  required  to  be  in- 
timately  familiar  with  the  device  under 
the  terms  of  the  contract,  it  would  appear 
that  in  many  instances  Government  is 
charged  twice  for  the  same  product,  namely 
the  instructor's  knowledge  of  device  op¬ 
eration  and  maintenance  procedures. 

No  satisfactory  explanation  has  been 
advanced  to  explain  why,  after  spending 
between  9  and  18  months  as  a  member  of 
the  team  which  develops  maintenance 
documentation  and  curriculum  data  items, 
an  instructor  would  require  anywhere  from 
12  to  36  weeks  to  prepare  to  teach  a  6 
week  course.  This  reasoning  is  even  more 
incongruous  when  it  is  realized  that  at 
least  3  of  the  6  weeks  will  be  devoted 
to  student  hands-on  training  in  the 
laboratory,  with  the  instructor  serving 
in  an  observer/advisor  capacity.  This 
leaves  only  3  weeks  of  actual  instruction 
time  for  which  preparation  is  required. 

DATA  ANALYSIS 

Maintenance  training  acquisition 
costs  are  defined  as  dollars  provided  to 
the  contractor  to  develop  and  present 
maintenance  training.  These  costs  in¬ 
clude  the  cost  of  instructor  services  and 
development  costs  for  curriculum  data 
items.  Because  of  the  infinite  number  of 
variables  present  in  maintenance  training 
acquisition  programs  which  extend  over  a 
period  of  two  to  three  years,  and  because 
of  the  variations  in  individual  contracts, 
precise  data,  especially  for  curriculum 
data  items  acquisition,  are  not  available. 
However,  from  the  accessible  data,  it  is 
possible  to  ascertain  that  maintenance 
training  costs  are  steadily  increasing. 

The  periods  chosen  for  this  study 
coincide  with  the  introduction  of  in¬ 
structional  systems  development  models  to 
the  maintenance  training  acquisition 
process  in  1973  -  1974,  and  the  issuance 
of  MIL-STD  1379A  in  1976.  Although  a 
portion  of  the  increased  costs  of  training 
acquisition  can  be  attributed  to  inflation, 
the  data  strongly  suggests  that  inappro¬ 
priate  curriculum  development  standards 
and  over-regulation  of  the  contractor's 
development  efforts  have  contributed 
greatly  to  this  increase. 

Curricula  data  items  are  not  normally 
priced  separately  in  a  contract  but  by  lot 
with  other  written  materials.  Available 
information  indicates  that  for  a  contract 
in  the  $2,000,000.  range,  the  average  cost 
of  training  data  items  is  $65,000.  or 
approx imately  3.3*  of  the  total  contract 
price.  As  has  been  shown  earlier,  many 
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of  these  data  items  are  unnecessary, 
thereby  adding  to  the  overall  non-cost 
effectiveness  of  the  maintenance  training 
acquisition  process. 

A  cost  analysis  of  Instructor  Services 
(preparation,  presentation,  and  other 
associated  costs)  indicates  a  major  in¬ 
crease  in  the  weekly  cost  of  maintenance 
training  beginning  in  1974.  The  average 
cost  per  week  for  the  period  1969  through 

1973  was  $2700.  while  for  the  period  1974 
through  1979,  the  average  cost  per  week 
was  $5100.  This  represents  an  increase 
of  93*.  A  second  comparison  shows  that 
the  average  cost  per  week  for  the  period 

1974  through  1976  was  $4600.,  while  for 
the  period  1977  through  1979,  costs 
averaged  $5700.  per  week,  a  24*  increase. 

A  comparison  of  the  average  costs  per 
week  for  the  1969  -  1973  period  with  the 
1977  -  1979  period  shows  an  increase  of 
111*. 

To  examine  maintenance  training 
costs  from  a  different  perspective  data 
was  gathered  from  a  random  sample  of  30 
maintenance  training  acquisition  programs; 
15  from  the  period  1972  -  1975,  and  15 
from  the  period  1976  -  1979.  The  data 
gathered  from  the  first  period  were  used 
in  establishing  a  baseline  with  which  to 
compare  the  costs  of  procurements  under 
MIl-STD  1379A,  which  was  issued  in  1976. 

A  summation  of  the  gathered  data 
indicates  that  in  the  period  1972  -  1975, 
220  weeks  of  maintenance  training  were 
procured.  Total  procurement  costs  for 
the  15  training  programs  were: 

(Costs  shown  in  millions  of  dollars) 

Ma intenance 

Contract  Hardware  Training 

31.8  23.4  .70 

In  comparison,  during  the  period  1976  - 
1979,  156  weeks  of  maintenance  training 
were  procured.  Total  procurement  costs 
for  the  15  training  programs  were: 

(Costs  shown  in  millions  of  dollars) 

Maintenance 

Contract  Hardware  Training 

27.9  20.8  .92 

The  average  cost  per  maintenance  training 
hour  for  the  1972-1975  programs  was  $80. 
For  the  post  1975  programs  the  average 
cost  per  maintenance  training  hour  was 
$147.  This  is  an  increase  of  $67.  or 
84*  per  training  hour. 

Because  of  the  problems  involved  in 
attempting  to  correct  for  the  effects 
of  inflation,  data  were  primarily 
analyzed  as  ratios  of  maintenance  training 
costs  to  total  contract  and  hardware 
costs : 


Hardware 

Maint. 

Maint. 

to 

Trng.  to 

Trng.  to 

Contract 

Contract 

Hardware 

Basel ine 
1972-1975 

73.5* 

2.2* 

3.0* 

Current 

1976-1979 

74.9* 

3.3* 

4.4% 

During  the  1972  -  1975  period  main¬ 
tenance  training  costs  were  2.2*  of  total 
contract  costs.  During  the  1976  -  1979 
period,  maintenance  training  costs  rose  to 
3.3*  of  the  total  contract  cost,  an  in¬ 
crease  of  50*.  The  maintenance  training 
to  hardware  cost  ratio  has  also  risen  by 
47*,  although  hardware  costs  have  in¬ 
creased  by  a  ratio  of  only  1.9*.  Results 
similar  to  these  have  been  found  in  a 
separate  study  recently  conducted  on  20 
other  maintenance  training  programs.  (4) 

The  last  decade  has  seen  advances  in 
maintenance  aids  such  as  software  diag¬ 
nostics  for  troubleshooting,  more  modu¬ 
larized  equipment  which  calls  only  for 
isolation  to  board  level,  and  card  testers 
which  expedite  and  facilitate  component 
isolation.  Still,  maintenance  training 
programs  have  become  more  expensive  both 
in  true  dollar  value  and  as  a  percentage 
of  contract  cost. 

RECOMMENDATION 

Fresh  thinking  is  sorely  needed  in 
the  general  area  of  life  cycle  device 
maintenance  and  the  specific  area  of 
maintenance  training  if  the  above  trends 
are  to  be  reversed.  Although  many  argu¬ 
ments  can  be  made  in  support  of  current 
methods  of  maintenance  training  acquisi¬ 
tion,  little  data  exists  to  support  these 
arguments.  It  must  also  be  remembered 
that  cost  effectiveness  and  training 
effectiveness  are  not  synonomous  terms. 

(8)  Even  if  it  could  be  proven  that 
current  acquisition  procedures  are 
effective,  this  does  not  exclude  the  very 
strong  probability  that  other  procedures 
may  be  equally  effective  and  may  cost 
considerably  less. 

Contractor  developed  maintenance 
training  may  be  acquired  more  cost  effec¬ 
tively  through  the  implementation  of  a 
Criterion  Referenced  Systems  Approach 
(CRSA)  to  maintenance  training  acquisition. 
This  approach  is  based  on  three  premises: 

a.  maintenance  training  is  an  in¬ 
tegral  part  of  device  procure¬ 
ment,  and  the  contractor  should 
be  tasked  with  the  full  and  total 
responsibility  for  providing 
training  that  will  produce  quali¬ 
fied  device  maintenance  personnel. 

b.  given  the  proper  motivation,  con¬ 
tractors  can  acquire  the  resources 
and  expertise  required  to  provide 
high  quality  maintenance  training 
in  a  cost  effective  manner. 

c.  Government  can  provide  this 
motivation  by  stipulating  in  the 
contract  that  a  significant 
portion  of  total  payment  will  be 
withheld  until  satisfactory 
training  is  received  and  by 
allowing  the  contractor  to  accom¬ 
plish  this  without  the  burden  of 
excessive  Government  restrictions 
and  directions. 
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CRSA  in  the  training  acquisition 
process  consists  of  three  components: 
INPUT-PROCESS-OUTPUT.  Neither  the  CRSA 
model  nor  the  CRSA  concepts  are  new  or 
unknown  in  the  training  field.  What  may 
be  unique  is  that  CRSA  is  based  on  the 
premise  that  if  Government  furnishes  the 
required  input  and  defines  the  required 
output,  and  holds  industry  totally  ac¬ 
countable  for  the  process,  high  caliber, 
cost  effective  maintenance  training  will 
resul t . 

The  INPUT  component  is  the  responsi¬ 
bility  of  Government.  Skill  and  know¬ 
ledge  profiles  have  been  developed  for 
each  Navy  rate  and  rating.  It  is  up  to 
Government  to  determine  from  what  skill 
and  knowledge  level  the  contractor  must 
start  developing  his  training.  Pro¬ 
spective  students  should  be  pretested 
to  ensure  that  Government  and  contractors 
are  aware  of  the  actual  skill  and  know¬ 
ledge  levels  of  these  students.  If  re¬ 
mediation  is  required,  it  may  prove  more 
cost  effective  to  procure  remedial  train¬ 
ing  through  some  other  source  than  to 
have  the  contractor  develop  a  longer 
training  course  to  accomodate  deficiencies 
in  student  input  levels.  This  component 
also  contains  performance  objectives  which 
have  been  derived  from  a  training  re¬ 
quirements  analysis.  (11)  Again,  it  is 
Government's  responsibility  to  determine 
what  level  of  performance  is  required  of 
a  course  graduate  and  to  define  these 
requirements  clearly  and  succinctly. 

The  ultimate  cost  effectiveness  of 
the  training  will  depend  in  large  part  on 
what  skill  and  knowledge  levels  have  been 
selected,  on  how  well  training  require¬ 
ments  have  been  analyzed,  and  on  how 
skillfully  performance  objectives  have 
been  constructed.  In  effect.  Government 
is  establishing  criterion  referenced  in¬ 
puts  and  certifying  that  the  contractor 
should  base  his  cost  proposal  on  these 
inputs.  If  for  some  reason  Government 
should  deviate  from  these  inputs,  the 
contractor  can  easily  measure  the  de¬ 
viation  and  rightfully  claim  a  commensu¬ 
rate  amount  of  consideration. 

The  PROCESS  component  is  the  sole 
responsibility  of  the  contractor.  After 
receiving  all  the  required  input  data, 
the  contractor  is  tasked  with  the  total 
freedom  to  develop  a  training  curriculum 
that  will  satisfy  the  performance  ob¬ 
jectives.  Since  satisfactory  completion 
of  training  will  be  based  on  criterion 
referenced  performance  objectives  and 
measurements,  it  becomes  the  responsi¬ 
bility  of  the  contractor  to  develop  an 
accurate  cost  proposal.  If  the  con¬ 
tractor  underestimates  the  effort  in¬ 
volved,  he  will  no  longer  be  able  to 
claim  his  problems  have  been  caused 
by  Government  demands  and  changes,  as 
currently  occurs  all  too  often.  Since 
there  will  be  no  provisions  for  "get  well" 
clauses  at  Government's  expense  In  the 
contract,  contractor  mistakes  or  inepti¬ 
tude  will  affect  only  the  contractor.  He 


will  still  be  required  to  provide  what  he 
said  he  would  or  suffer  the  loss  of  a 
significant  portion  of  the  overall  con¬ 
tract  price. 

It  is  in  this  component  of  the  CRSA 
that  strict  adherence  to  the  directives 
and  instructions  cited  earlier  is  re¬ 
quired.  SECNAVINST  4200. 27A  is  specific 
in  defining  the  required  Government/con¬ 
tractor  relationship: 

The  Government  may... obtain  the 
(required)  work  by  contract,  pro¬ 
viding  two  conditions  are  met: 

(1)  the  contract  itself  must  ask 
for  the  finished  product,  only,  and 

(2)  the  contract  must  be  adminis¬ 
tered  in  such  a  way  that  control 
and  supervision  over  the  work  and 
discretion  as  to  the  techniques 
which  will  be  used  remain  solely 
with  the  contractor. . . . 

The  intent  of  this  statement  is  made  ex¬ 
plicit  in  the  very  next  sentence. 

...In  other  words,  if  the  Govern¬ 
ment  wants  a  building  painted  (or 
personnel  trained),  it  defines  the 
job,  lets  the  contractor  paint  the 
building  (train  the  personnel)  as 
he  sees  fit,  and  then  accepts  it  or 
rejects  it  solely  on  the  basis  of 
whether  the  completed  job  meets  the 
contract  specification.... 

The  OUTPUT  component  is  the  respon¬ 
sibility  of  Government,  which  must  de¬ 
termine  what  it  expects  the  student  to 
be  able  to  accomplish  upon  completion  of 
maintenance  training.  These  expectations 
must  be  stated  in  terms  of  effective  per¬ 
formance  as  established  by  criterion 
based  performance  objectives.  In  order 
to  accomplish  this,  a  thorough  maintenance 
training  requirements  analysis  must  be 
commenced  as  soon  as  the  engineering 
specifications  are  formulated.  From  this 
analysis,  functional  training  specifica¬ 
tions  are  written  by  Government.  These 
specifications  set  forth  the  performance 
objectives  and  assessment  methods  which 
Government  intends  to  use  to  accept  or 
reject  the  contractor's  training  effort. 

The  performance  objectives  and  methods 
of  assessment  become  part  of  the  input 
component.  Once  these  items  are  defined. 
Government  is  able  to  write  an  RFP  which 
will  enable  contractors  to  develop  effec¬ 
tive,  competitively  priced  cost  proposals. 
After  a  proposal  has  been  accepted  and  a 
contract  awarded,  the  data  therein  is 
used  to  develop  a  definitive  assessment 
design  and  specific  performance  measure¬ 
ments  with  which  to  measure  the  con¬ 
tractor's  maintenance  training  performance. 

The  preferred  method  of  assessment  is 
a  comprehensive  performance  examination 
which  might  also  serve  as  a  maintainability 
demonstration.  Such  an  assessment  instru¬ 
ment  would  assure  that  the  trainees  are 
actually  capable  of  fully  maintaining  the 
device  to  the  level  required  by  the  con¬ 
tract.  If  a  written  assessment  is  deemed 
desireable,  the  Solomon  design  is  an 
excellent  method  of  determining  the  value 
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of  the  instruction  received  in  compari¬ 
son  to  the  general  knowledge  that  the 
trainee  may  posess  from  experience  or 
by  association  with  other  training  de¬ 
vices  . 

SUMMARY 

The  rising  costs  of  contractor  de¬ 
veloped  maintenance  training  are  de¬ 
creasing  the  cost  effectiveness  of 
current  training  acquisition  procedures. 
Given  the  austere  funding  climate  of  the 
predictable  future,  this  situation  is 
intolerable.  Preliminary  research  in¬ 
dicates  that  several  underlying  causes 
contribute  to  the  unwarranted  escalation 
of  maintenance  training  costs.  When 
these  causes  are  examined,  it  becomes 
apparent  that  adherence  to  inappropriate 
acquisition  policies  results  in  in¬ 
adequate  definition  of  needs,  improper 
utilization  of  Government  personnel, 
unnecessary  procurement  of  data  and 
services,  and  loss  of  contractor  in¬ 
genuity  and  resourcefulness. 

Obviously,  there  is  no  one  perfect 
method  which  will  ensure  cost  effective 
acquisition  of  maintenance  training. 

The  method  recommended  in  this  paper  is 
just  as  susceptible  to  inept  application 
as  is  the  current  method.  However,  the 
authors  believe  that  given  equal  amounts 
of  conscientious  attention  by  mainten¬ 
ance  training  acquisition  personnel, 
the  Criterion  Referenced  Systems  Approach 
will  prove  more  cost  effective  than  have 
the  current  methods  of  maintenance 
training  acquisition. 
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ABSTRACT 

This  paper  suggests  that  data  management  acquisition  procedures  used  by  Government  are 
often  not  cost  effective.  All  sorts  of  military  standards  are  deemed  “the  thing"  when 
contractors'  formats  would  suffice  at  a  substantial  savings.  Data  management  acquisition 
procedures  need  an  overhaul.  The  Naval  Training  Equipment  Center  (NAVTRAEQUIPCEN)  has 
developed  a  Recommended  List  of  Approved  Data  Item  Descriptions  (DID* s)  and  DID  numbers  for 
a  major  weapon  buy.  This  list  must  be  tailored  to  the  requirement.  In  addition.  Data 
Management,  Technical  (DAMTEC),  an  automated  system  for  tracking  the  Contract  Data  Require¬ 
ments  List  (CDRL)  Items  In  contracts,  has  been  developed  to  monitor  the  progress  of  the 
deliverable  data  and  to  serve  as  a  data  bank  for  use  by  project  team  members.  This  data 
bank  provides  cost  comparison  Information  based  upon  cost  of  data  previously  bought  for 
various  training  equipment. 

INTRODUCTION 


During  the  1 960 ' s  the  National  Aeronautics  Space 
Administration  (NASA)  bought  tralnloads  of  data, 
and  developed  a  slide  rule  for  assessing  Its  cost. 
Just  as  NASA  was  concerned  by  the  mounting  cost  of 
documentation,  we  at  NAVTRAEQUIPCEN  are  also  con¬ 
cerned.  Sometimes  It  appears  that  all  Government 
wants,  or  Is  looking  for.  Is  more  and  more  techni¬ 
cal  data.  "Not  so,"  says  NAVTRAEQUIPCEN.  In  1979 
the  Center  sharpened  Its  data  management  pencil 
and  "managed"  to  delete  31  Items  from  an  81-Item 
list  of  data  for  larger  procurements  without 
weakening  support  to  the  related  training  equip¬ 
ment.  Cost  savings  have  been  dramatic. 

Definition  of  Technical  Data 

According  to  DOD,  technical  data  are  “recorded 
Information  used  to  define  a  design  and  to  produce, 
support,  maintain,  or  operate  Items  of  defense 
material."  These  data  may  be  recorded  as  graphic 
or  pictorial  delineations  In  media  such  as  draw¬ 
ings  or  photographs;  text  In  specifications  or 
related  performance  or  design-type  documents;  in 
machine  forms  such  as  punched  cards,  magnetic  tape, 
computer  memory  printouts;  or  may  be  retained  In 
computer  memory.  Examples  of  recorded  Information 
Include  engineering  drawings  and  associated  lists, 
specifications,  standards,  process  sheets,  manuals, 
technical  reports,  catalog  Item  Identifications, 
and  related  Information.  These  technical  data  are 
Identified  on  a  DD  Form  1423,  Contract  Data 
Requirements  List  (CDRL),  If  the  contract  requires 
delivery  of  the  data. 

Data  Cost  Factors 

The  contractor  has  to  prepare  certain  data  to 
satisfy  the  design,  development,  testing,  and 
production  aspects  of  the  contract  regardless  of 
the  deliverable  data  requirements  placed  on  the 
contract  by  the  government.  Knowing  that  much  of 
the  data  required  by  the  government  are  basically 
the  same  as  that  prepared  by  the  contractor  for 
his  own  use  In  satisfying  the  contract,  DOD,  In 
conjunction  with  Aerospace  Industry  Association, 
developed  what  Is  known  as  the  "over-and-above" 
concept. 


This  concept  In  action  requires  the  contractor 
to  Include  in  his  estimated  selling  price  on  each 
specified  data  Item,  that  portion  of  the  develop¬ 
ment/preparation  data  effort  expended  by  him  to 
solely  satisfy  the  government  requirement  for  a 
deliverable  data  Item. 

This  estimated  selling  price  to  the  government 
also  Includes  General  and  Administrative  (G&A) 
costs,  overhead,  and  profit.  Under  the  "over-and- 
above"  concept  the  estimated  selling  price  Is  con¬ 
tingent  upon  whether  or  not  the  contractor  had  to 
prepare  the  data  for  his  own  use  or  solely  to  meet 
the  government  requirements. 

Current  DOD  policy  Identifies  four  pricing 
conditions,  known  as  "Price  Groups,"  which  the 
contractor  utilizes  under  the  "over-and-above" 
concept.  They  are  sunmarlzed  as  follows  and  then 
defined  In  the  narrative. 


Group 

Essential  to 
Performance 
of  Contract 

Added 

Effort 

Required 

Price 

I 

NO 

YES  (100*) 

YES 

II 

YES 

YES 

YES 

III 

SOME 

SOME 

YES 

IV 

NONE 

NONE 

NO 

Group  I  -  Data  which  the  contractor  prepares  to 
satisfy  the  Government.  The  contractor  does  not 
need  this  type  of  data  to  perform  the  rest  of  the 
contract.  Price  would  be  based  on  Identifiable 
direct  costs,  overhead,  and  profit.  Technical 
manuals  prepared  for  government  use  are  an  example. 
(Engineering  drawings  generally  are  the  source 
material  for  technical  manuals.  The  cost  of 
developing  these  drawings  Is  not  a  cost  of  produc¬ 
ing  the  manuals.) 

Group  II  -  Data  essential  to  contract  performance 
which  must  be  reworked  or  amended  to  conform  to 
government  requirements.  The  price  for  data  In 
this  group  would  be  based  on  the  direct  cost  to 
conform  the  original  data  to  government  needs  and 
to  deliver  It,  plus  allocable  overhead  and  profit. 
Much  of  the  data  purchased  Is  from  this  group. 
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Group  III  -  Data  which  the  contractor  must  develop 
for  nls  own  use  ant!  which  require  no  substantial 
change  to  conform  to  government  requirements 
regarding  depth  of  content,  format,  frequency  of 
submittal,  preparation,  and  quality  of  data.  Only 
the  cost  of  reproducing,  handling,  and  delivery, 
plus  overhead  and  profit,  are  considered  In  pric¬ 
ing  data  In  this  group.  An  example  of  the  kind  of 
data  to  be  categorized  as  Group  III  would  be 
D0D-D-1000  Level  1  drawings  used  In  the  manufac¬ 
turers  normal  plant  functions.  Level  1  drawings 
are  those  drawn  to  company  standards. 

Group  IV  -  Data  which  the  contractor  has  developed 
as  part  of  his  commercial  business.  Not  much  of 
these  data  are  required  and  the  cost  Is  Insignifi¬ 
cant.  The  Item  Is  coded  "no  charge."  Example: 

A  brochure  or  brief  manual  developed  for  commer¬ 
cial  application  which  will  be  acquired  In  small 
quantities,  the  added  cost  Is  too  small  to  justify 
the  expense  of  computing  the  charge  that  otherwise 
would  go  with  the  acquisition. 


Data.  Data, 


xhere?  —  What  to  Buy? 


Prior  to  1979,  NAVTRAEQUIPCEN  was  being 
Inundated  with  data.  On  a  typical  major  procure¬ 
ment,  "minimum"  data  requirements  totalled  81 
Items.  The  Center's  Data  Management  Officer 
chaired  a  data  review  board  which  screened  and 
rewrote  all  Data  Item  Descriptions  (DID's)  to 
reduce  cost-driving  requirements,  and  encouraged 
use  of  contractor's  format.  The  list  was  reduced 
to  50  Items.  The  Imnedlate  Impact  was  to  cut  the 
cost  of  data  on  a  single  procurement  from  $436,223 
to  $309,709.83  for  a  savings  of  $137,513.17.  The 
new  "Recomnended  Data  List"  Is  shown  In  figure  1. 
It  Is  a  maximum  requirement  list  for  a  major 
training  equipment  procurement,  and  only  selected 
DID's  are  expected  to  be  used  to  acquire  necessary 
data;  tailoring  should  take  place  throughout  the 
system/equipment  life  cycle. 


Exhibit  A  -  Engineering 


(Hardware) 


Trainer  Engineering  Design  Report 
Trainer  Test  Pi-ocedures 
Trainer  Facilities  Report 
Trainer  Photographs 
Trainer  Slides 
Trainer  Mockup  Report 
Trainer  GFE  Report 
Trainer  Artist  Sketch 


UDI-S-25591A 

UDI-T-25594A 

UDI-P-25579 

UDI-E-25559 

UDI-E-25560 

UDI-E-25565 

UDI-P-25581 

UDI-E-25562 


Exhibit  A  -  Engineering 


(Software) 


9.  Software  Development  Plan  (SDP) 

10.  Specification,  Program  Performance  (PPS) 

11.  Interface  Design  Specification  (IDS) 

12.  Computer  Program  Test  Plan 

13.  Computer  Program  Test  Procedures/Reports 

14.  Program  Design  Specification  (PDS) 

15.  Data  Base  Design  Document  (DBDD) 

16.  Program  Package  Document  (PPD) 

17.  Program  Description  Docunent  (PDD) 

18.  Operator's  Manual 

19.  Computer  Software  Trouble  Report  (STR) 

20.  Software  Quality  Assurance  Plan 

21.  Software  Configuration  Management  Plan 

22.  Software  Change  Proposal  (SCP)  and  Software 

Enhancement  Proposal  (SEP) 

Exhibit  8  -  Administrative  Data 

23.  Trainer  Technical  Progress  Report 

24.  Cost  Schedule  Status  Report 

Exhibit  C  -  Provisioning  Data 

25.  Provisioning  Parts  List  (PPL) 

26.  Provisioning  Forms 

27.  Provisioning  Parts  List  Index  (PPLI) 

28.  Supplementary  Provisioning  Technical 

Documentation  (SPTD) 

29.  Provisioning  and  other  Procurement  Screening  Data 

30.  Repairable  Item  List  (RIL) 

31.  Post  Conference  Provisioning  List  (PCPL) 

32.  Interim  Support  Item  List  (ISIL) 

33.  Recomnended  List  of  Maintenance,  Test  and  Support 

Equipment  (MT&SE) 

34.  Design  Change  Notice  (DCN) 

35.  Inventory /Util Izatlon  Data  Report  (IUDR)* 


DI-A-21 76/MOD 
DI-E-2136/M0D 
DI-E-21 35/MOD 
DI-T-2142/MOD 
DI-T-21 44/MOD 
DI-E-21 38/MOD 
DI-S-21 40/M0D 
DI-S-2141 /MOD 
DI-S-21 39/MOD 
DI-M-21 45/MOD 
DI-E-21 78/MOD 
DI-R-21 74/MOD 
DI-E-21 75/MOD 

DI-E-21 77/MOD 


UDI-A-25602A 

DI-F-6010 


DI-V-7002 

DI-V-2022/M0D 

DI-V-7000 

DI-V-7016B 

DI-V-7005 

DI-V-2172 

DI-V-7006 

DI-V-5275/M0D 

DI-V-7009 

U01-V-25513 


Exhibit  D  -  Maintenance  Data 

36.  Rellablllty/Malntalnablllty  Status  Reports 

37.  Nonstandard  Part  Approval  Requests 

36.  Integrated  Logistic  Support  Plan  (ILSP) 

39.  Training  Equipment  Stannary 

40.  Contractor  Field  Service  Reports 

41.  Drawings,  Engineering  and  Associated  Lists 

42.  Trainer  ECP  Stannary 

43.  Training  Device  Inventory  Record 

Exhibit  E  -  Publications  Data 


44. 


45. 

46. 

47. 


(a)  Manual,  Technical,  Functionally  Oriented 
Technical  Manual  (FOTM)  For  Training 
Devices** 

(b)  Manual,  Technical,  Operation  and  Maintenance 
Instructions** 

PMS  Documentation 

Manual,  Technical,  Connerclal  Standard 
Vendor  Equipment  Data  Updates 


Exhibit  F  -  Training  Data 

48.  Outlines,  Training  Course  Curriculum  (Option  1  only) 

49.  Guides,  Instructor/Lesson-Training  Courses 

50.  Audiovisual,  Aids,  Master  Reproduclbles  and  Review 

Copies  Training  Equipment  and  Training  Courses 

51.  Test,  Measurement  of  Student  Achievement 

52.  Forms  Evaluation  -  Student  &  Training  Course 

53.  Handbook,  Instructor’s  Utilization  Simulation 

Equipment 

54.  Guides,  Students  -  Training  Courses 

•Not  required  If  3M  reporting  Invoked. 

••Only  one  of  these  Items  Is  used. 


UDI-R-255844/ 

UDI-L-25572A 

DI-E-7028 

UDI-L-25622B 

UDI-L-25510 

UDI-L-25514C 

DI-E-7031/M0D 

UDI-E-25603 

UDI-L-25578A 


UDI-M-25597B 


UDI-M-25575 

UDI-L-20304A 

UDI-M-25576 

UDI-M-25700A 


DI-H-2026 

DI-H-2073 

DI-H-2122 

DI-H-2033A 

DI-H-2165 

DI-H-2028A 

DI-H-2102A 


Figure  1.  Recomnended  Data  List 


DAHTEC 


DATA  ELEMENT  DICTIONARY 


To  monitor  the  progress  status  of  these  DIO's, 
DAMTEC,  an  automated  data  processing  (ADP)  system 
was  developed.  This  unique  ADP  application 
monitors  the  progress  of  the  deliverable  data 
Items  Identified  on  DD  Form  1423's  In  the  contract. 
DAMTEC  serves  as  a  data  bank  for  use  of  project 
team  members  making  them  aware  of  the  total 
picture.  In  addition,  DAMTEC  alerts  the  team  when 
the  delivery  of  a  CDRL  Item  Is  due,  or  when  the 
delivery  of  a  CDRL  Item  Is  overdue.  This  system 
provides  up-to-date  and  accurate  access  to  all 
Information  related  to  contract  technical  data. 

On  a  weekly  basis,  one  report  automatically  flags 
those  contract/item  numbers  that  have  potential 
slippage  of  milestone  dates.  The  DAMTEC  Element 
Dictionary  consists  of  33  data  elements,  nine  of 
which  are  automatically  extracted  from  two  other 
NAVTRAEQUIPCEN  computerized  systems,  the  Technical 
Program  Control  System  and  the  Contract  Status 
System.  In  this  way,  the  Data  Specialist  need  not 
duplicate  effort  by  Inputting  like  data  which  Is 
already  stored  In  the  computer. 


ACCEPTANCE  DATE 
APPROVAL  COOE 
ACTUAL  RESPONSE  DATE 
CANCEL  DATE 
COMPLETION  DATE 
CONTRACT  AWARD  VALUE 
CONTRACT  NUMBER 
CONTRACTOR  NAME 
CONTRACT  STATUS 
CONTRACTOR  RESPONSE  DUE 
DATE  RECEIVED 
DUE  DATE 

DEVICE  NOMENCLATURE 
DEVICE  NUMBER 
DID  TITLE 
ELEMENT  MANAGER 
EXHIBIT  IDENTIFIER 
FREQUENCY 
FINAL 


GOVERNMENT  RESPONSE  DUE 
IF  ITEM  WAS "ACCEPTED/ 
REJECTED 
ITEM  DATA  COST 
ITEM  NUMBER 
LATEST  MILESTONE  DATE 
NATIONAL  STOCK  NUMBER/ 
LOCAL  STOCK  NUMBER 
OBLIGATION  ACTUAL  DATE 
PRELIMINARY 
RECORD  CHANGE  DATE 
PROGRAM  CODE 
REMARKS 
TASK  NUMBER 

TECHNICAL  DATA  SPECIALIST 
TECHNICAL  OFFICE 
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From  the  Data  Element  Dictionary  comes  the  "DAMTEC 
Data  Item  Status  Report  by  Contract  Nurter".  This 
report  selects  all  record  on  the  DAMTEC  Data  Base, 
except  those  coded  KC"  (closed)  In  Contract  Status 
data  element.  Another  report,  “Milestones  Due/ 
Overdue,”  selects  all  records  with  the  latest  mile¬ 
stone  past  due  or  due  within  seven  days  of  the  “as 
of"  date  on  the  report.  One  of  the  most  used 
reports  Is  the  “Data  Item  Status  Report  by  Techni¬ 
cal  Office  and  Element  Manager".  The  element 
manager  has  a  composite  of  what  line  Items  he  Is 
responsible  for  reviewing  and.  If  It  has  already 
been  received,  the  response/ review  date  he  must 
meet.  AD  HOC  reports,  selecting  any  Imaginable 
combination  from  the  Random  Select  Report  Module, 
can  be  requested  by  properly  querying  the  data  bank 
and  asking  for  contract  number  sequence,  device 
number  sequence,  DID  sequence,  etc.  As  Indicated 
above,  the  primary  purpose  of  DAMTEC  Is  to  serve 
as  a  progress  monitoring  tool.  However,  this 
data  bank,  because  It  can  store  and  report  data 
Item  costs,  also  serves  as  an  excellent  guide  In 
determining  Item  "should  costs",  and  otherwise  aid 
In  tailoring  requirements  for  new  procurements. 
Although  DAMTEC  has  been  In  operation  for  less  than 
a  year.  It  has  been  praised  by  project  team 
members  and  element  managers.  New  uses  for  DAMTEC 
are  constantly  being  discovered,  and  NAVTRAEQUIPCEN 
sees  It  growing  Into  a  bigger  and  better  project 
management  tool. 


CONCLUSIONS 

Guidelines  should  be  developed  for  selecting 
tailoring,  and  applying  specifications  and 
standards,  data  requirements,  and  management 
systems  to  assist  personnel. 


Cost-driving  requirements  of  specifications 
should  be  Identified  and  tailored  prior  to 
requesting  proposals. 

Data  banks  should  be  established  and  used  In 
modifying  requirements  for  new  procurements. 

Industry  should  assist  the  Government  In 
Identifying  the  cost-driving  requirements  of 
present  DID's  and  standards  Imposed  on  Government 
procurements. 

Industry  should  automatically  provide  alter¬ 
nate  proposals  on  all  RFP's  to  Indicate  how  con¬ 
tractor  format  or  other  alternatives  can  be  used 
to  reduce  proposal  cost  with  no  loss  In  technical 
Information  being  provided. 
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EW  TRAINING  USING  RADAR  ELECTROMAGNETIC  ENVIRONMENT 

SIMULATION 

John  F.  Michaels,  Vice  President  of  Marketing 
and 

William  W.  Kestenbaum,  Sr.  Systems  Engineer 
Republic  Electronics,  Inc. 

ABSTRACT 

This  paper  describes  what  is  necessary  in  a  simulator  to  achieve  a  high  degree  of  operator  com¬ 
petence  in  interpreting  a  radar  display  in  a  threat  environment.  The  achieved  level  of  operator  com¬ 
petence  is  a  function  of  the  degree  of  realism  and  complexity  created.  The  models  used  for  the  gen¬ 
eration  of  the  signatures  of  aircraft/missile/ship  targets,  chaff,  sea  clutter,  rain  and  jammers, 
operating  in  a  dynamically  controlled  environment,  are  described  in  detail.  The  hardware /software 
implementation  and  equipment  installation,  together  with  problems  encountered  during  the  simulator 
design  are  also  discussed. 


INTRODUCTION 

Today's  electronic  technology  can  provide 
a  realistic,  accurate,  and  extremely  flexible 
simulation  of  the  electromagnetic  environment 
compatible  with  the  most  sophisticated  2D  and 
3D  weapon  control  and  surveillance  radar  sys¬ 
tems.  This  ability,  to  create  on  command,  a 
simulation  of  a  dynamic  multiple-target /threat 
environment,  including  clutter  and  weather  ef¬ 
fects,  allows  for  the  performance  of  radar  op¬ 
erator  training  to  a  degree  previously  unachiev¬ 
able.  Instant  recognition  of  and  response  to 
multiple  threats  is  crucial  to  mission  success  - 
and  often  to  crew  survival. 

The  key  lies  in  tt  •:  -Jity  to  simulate  radar 
returns  and  jam  rer  s-gw  :ures  with  a  degree  of 
realism  not  differentiable  from  real  world  radar 
inputs.  The  equipment  must  provide  in  real  time 
that  portion  of  an  electromagnetic  environment 
comprised  of  .multiple  airborne  targets,  multiple 
chaff  events  dispensed  by  the  simulated  targets, 
clutter,  weather,  and  ECM  waveforms  emanating 
from  selected  simulated  targets.  The  ECM  wave¬ 
forms  should  be  capable  of  both  denial  and  con¬ 
fusion  techniques.  In  addition,  the  simulator 
must  be  coordinated  with  the  real  time  operation 
of  the  radar  with  respect  to  radar  pointing  direc¬ 
tion,  range  sweep  timing,  and  transmitted  fre¬ 
quency  and  phase.  This  modeling,  therefore, 
should  include  accurate  simulation  of: 

1.  Signal  strength  variation  due  to  target 
radar  cross  section  (RCS)  fluctuation. 

2.  Slant  range  and  radar  transmitter  power. 

3.  Modulation  due  to  target  position  in  the 
antenna  pattern  (main,  side,  and  back  lobes.) 

4.  Doppler  shift  and  doppler  noise. 

THE  TECHNICAL  PROBLEM 

In  order  to  create  an  illusion  to  the  radar  of 
actual  radar  returns  it  is  important  to  consider 
the  many  signal  parameters  and  radar  interfaces. 


1  would  like  to  briefly  describe  these  radar  para¬ 
meters  and  then  dig  a  little  deeper  into  the  di¬ 
mensions  of  these  parameters: 

Radar  Cross  Section 

Scintillation  Noise  and  Other  Noise  Compo¬ 
nents 

Target  Dynamics 

Path  Loss  Factors 

Doppler 

Scan  Modulation  and  Antenna  Gain 

Extended  Targets  (Chaff,  Rain.  Sea  Clutter) 

Jammers 

Radar  (2D  &  3D)  and  Operator  Interface 
Radar  Cross  Section  (RCS) 

The  RCS  model  should  be  the  result  of  a 
simple  operation  input  of  mean  cross  section 
over  a  range  that  considers  all  sizes  of  expected 
encountered  targets. 

Scintillation  Noise  and  Other  Noise  Components 

The  model  should  include  phase /amplitude 
fluctuation  in  regard  to  target  aspect  angle,  am¬ 
plitude  noise  modulation  to  simulate  target  cross 
section  fluctuation  and  doppler  frequency  noise 
modulation  to  simulate  target  doppler  frequency 
noise  fluctuation. 

Target  Dynamics 

The  target  model  should  be  capable  of  sim¬ 
ulating  both  maneuverable  and  fixed  targets  in 
three  dimensions,  (range,  azimuth,  and  altitude.) 
The  maneuverable  targets  should  be  capable  of 
linear  and  circular  motion  and  of  having  a  rate  of 
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ascent  or  descent  applied  individually. 

Path  Loss  Factors 

All  losses,  whether  one  way  or  two  way, 
should  be  modeled  such  as;  range  loss,  frequency 
loss,  chaff  and  rain  attenuation,  over  a  dynamic 
range  and  resolution  compatible  with  expected 
target  and  jammer  conditions. 

Doppler 

Target  doppler  frequency  should  be  modeled 
dynamically  in  accordance  with  the  flight  path 
motion  of  each  target. 

Scan  Modulation  and  Antenna  Gain 

The  absolute  pointing  direction  of  the  antenna 
should  be  tracked  in  real  time  by  the  simulator. 
The  antenna  patterns  should  be  used  to  provide 
coordinated  amplitude  modulation  for  all  modeled 
signals  with  respect  to  the  one  way  or  two  way 
antenna  gain  as  required. 

Extended  Targets  (Chaff.  Rain,  Sea  Clutter) 

Coherent  radar  returns  from  a  number  of 
extended  targets,  such  as  chaff,  rain,  and  sea 
clutter  should  be  modeled.  The  model  should 
consider  the  additive  effects  of  many  scatterers, 
air  mass  motion  effects,  screening  effects,  and 
altitude,  and  in  addition,  should  provide  for  both 
amplitude  noise  modulation  to  simulate  cross  sec¬ 
tion  fluctuations  and  doppler  frequency  effects 
including  doppler  noise. 

Jammers 

The  simulator  should  be  capable  of  modeling 
a  number  of  threats  assignable  to  selected  tar¬ 
gets.  Denial  techniques  such  as  barrage  and 
spot  noise  and  swept  CW  should  be  available. 

In  addition,  noncoherent  types  of  confusion  tech¬ 
niques  such  as  range  deception  with  a  cover  pulse, 
inverse  gain,  synchronous  and  non-synchronous 
pulse  train  should  be  available.  The  coherent 
type  confusion  techniques  should  be  range  decep¬ 
tion  and  inverse  gain. 

Radar  Interface 

The  simulator  model  should  be  compatible 
with  both  2D  and  3D  radars  and  a  direct  sum¬ 
mation  with  the  real  world  radar  input. 

Operator  Interface 

A  simple  operator  interface  is  of  paramount 
importance.  Automatic  and  manual  data  entry 
should  both  be  considered.  In  addition,  a  means  of 
displaying  continuously  updated  data  on  all  tar¬ 
gets  should  be  considered. 


A  RADAR  ELECTROMAGNETIC 
ENVIRONMENT  MODEL 

Radar  Cross  Section 

Two  distinct  types  of  targets  are  considered, 
those  that  are  short  compared  to  a  radar's  pulse 
length  and  those  that  are  long. 

A  target  that  is  physically  short  effectively 
echos  a  near  exact  replica  of  the  transmitted 
signal.  Targets  that  are  long  include  rain,  sea 
clutter,  and  the  extended  chaff  drops.  For  long 
or  extended  targets,  the  radar  return  is  no  long¬ 
er  a  simple  replica  of  the  transmitted  Bignal, 
but  a  superposition  of  many,  many  returns 
from  individual  scattering  centers.  Extended 
target  returns  are  discussed  later  in  this  re¬ 
port.  The  RCS  assigned  to  each  type  of  target 
is  based  on  experimental  derived  means.  For 
example,  aircraft  typically  vary  anywhere  from 
0. 1  to  20  square  meters.  This  statistical  mean 
is  taken  as  a  point  of  departure  for  all  subse¬ 
quent  variation  and  is  the  operator  entry  into 
the  system. 

Scintillation  Noise  and  Other  Noise  Components 

The  signal  returned  to  the  radar's  antenna 
for  any  given  target,  even  if  physically  very 
short  is  known  to  vary  from  reply  to  reply.  To 
achieve  a  realistic  model,  the  more  significant 
contributions  must  be  considered. 

The  first  important  signal  strength  variation  is 
the  deviation  from  mean  RCS  due  to  irregulari¬ 
ties  of  the  target  itself.  Fig.  1  is  a  typical  size 
aircraft.  As  can  be  seen,  minor  changes  in 
aspect  angle  can  vary  reply  signal  strength  up 
to  20  dB,  yet  for  all  of  its  wild  variations,  the 
return  signal  strength  iB  essentially  free  of  step 


Experimental  cross  section  of  a  two  engine 
bomber  at  10  cm  wavelength  as  a  function  of 
azimuth  angle.  (SKOLNIK) 

When  considering  return  signal  strength 
variations  on  a  pulse  to  pulBe  basis  (PRI  of  2 
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milliseconds)  for  an  aircraft  in  a  "g"  limited  ma¬ 
neuver  the  variations  are  quite  minor.  On  the 
other  hand,  in  the  scan  to  scan  period  (perhaps 
5  to  10  seconds)  the  return  strength  could  vary 
40  dB.  We  thus  experience  a  high  degree  of  cor¬ 
relation,  pulse  to  pulse,  and  full  decorrelation 
of  signal  strength  scan  to  scan.  The  statistical 
behavior  of  the  decorrelation  is  well  defined  in 
literature  as  Swerling  Case  I 


P(<r>  =  I 

<r  ave. 


exp  (-o’  ) 
c^Bve. 


The  second  most  important  signal  strength 
variation  arises  from  multipath.  Figure  2  in¬ 
dicates  the  four  possible  multipaths  considered 
on  both  transmit  and  receive. 


Multipath  Signal  Lines 
Figure  2 


Depending  on  the  geometry  of  the  situation, 
Lne  major  variables  being  target  range  and 
elevation,  the  difference  in  path  length  can  change 
radically  with  changes  in  aircraft  position.  As 
the  radar  wavelength  for  L  and  S  boards  are  but 
30  and  10  cm  respectively,  the  aircraft  motion, 
even  under  the  modifying  condition  of  the  situa¬ 
tion  geometry,  results  in  significant  variation  of 
differential  path  length  on  a  pulse  to  pulse  basis. 


Path  one.  the  most  direct  one.  has  the  least 
attenuation.  Path  two  and  three  experience  atten¬ 
uation  from  a  single  reflection  but  are  both  of 
the  same  length.  Path  four,  the  longest  of  them 
all,  experiences  twice  the  bounce  loss  of  two  and 
three.  The  most  significant  signal  is  thus  path 
one,  of  nearly  equal  significants  the  combina¬ 
tion  of  two  and  three,  and  with  a  much  lesser 
effect  from  path  four. 


vary  from  the  norm  of  unity  (say  1  volt)  for  the 
direct  ray  (path  one)  to  a  maximum  of  2  Volts 
and  a  minimum  of  0  Volts.  Expressed  in  dB  we 

expect  a  variation  from  0  to  +6  dB  upward  to  - 
infinity  downward.  The  contribution  from  path 
four,  however,  prevents  the  zero  (-  infinity  dB) 
and  -20  dB  is  a  realistic  approximation. 

The  overall  effect  of  this  multipath  problem 
is  the  rapid  fluctuation  of  signal  strength  on  a 
pulse  to  pulse  basis,  with  but  little  variation 
of  the  mean  on  a  scan  to  scan  basis. 

Again,  the  statistics  of  this  problem  are 
well  described  in  literature,  using  the  same 
equation  as  above,  but  classified  as  Swerling  II. 

The  discussion  above  gave  a  basis  for 
extent  of  the  distributions  used  based  on  physical 
behavior.  The  slow  variations  can  yield  nulls 
down  to  -40  dB  on  occasion  while  the  fast  one 
generally  are  best  described  by  a  distribution 
between  the  limits  of  +6  and  -20  dB. 

In  literature  Swerling  I  &  n  are  often  used  as 
two  distinct  effects,  some  occuring  at  some  time 
under  some  circumstances,  and  the  other  under 
some  other  conditions.  Data  recordings  taken 
in  the  past  on  the  AN/FPS  35.  AN/TPS34,  and 
others  have  convinced  us  that  a  real  life  situation 
involves  both  effects,  i.e.  superposition  of 
both  a  Swerling  I  and  a  Swerling  II  case. 

The  fast  variations,  with  a  range  of  +6  to 
-20  dB  (based  on  a  quasi  correlated  statistically 
normal  distribution  in  voltage,  not  power, )  are 
combined  with  the  slow  near  random  variations, 
with  prescribed  distribution  in  the  range  from  0 
to  -40  dB.  Special  provisions  were  made  to  in¬ 
sure  that  correlating  data  is  correlated  for 
replies  from  any  one  target,  yet  be  decorrelated 
from  another  target,  though  within  the  same 
radar  beam. 

Since  the  variations  of  correlating  data  are 
time  dependent,  the  degree  of  change  is  based 
on  elapsed  time  only  and  the  radar  PRI  does 
not  enter  the  generation  of  correlated  noise. 

Target  Dynamics 


In  the  model  used,  path  one  has  been  assigned 
near  equal  significance  with  the  combination  of 
two  and  three. 


In  order  to  create  a  dynamic  scenario  of  multi 
pie  target/threat  events  a  number  of  entered 
parameters  must  be  acted  on: 


This  near  equivalence  of  path  one  and  the 
sum  of  two  and  three,  were  amply  demonstrated 
by  the  naval  tracking/tests  run  at  Chesapeake 
Bay  Annex  during  tracking  experiments  for  low 
flying  aircraft. 

As  signals  in  space  add  on  a  voltage  basis, 
and  aa  the  two  significant  signals  can  be  at  any 
phase  with  equal  probability,  the  addition  can 


Target  Number  -  Aircraft  Identifier 
Initial  Position  -  Range,  Azimuth,  Altitude 
Radar  Cross  Section  -  Mean  Cross  Section  in 
Meters'* 

Target  Dynamics  -  Heading,  Velocity,  Turn  Rate 
(Right  or  Left),  Elevation  Rate  (Ascent  or  De¬ 
scent) 

Designated  Jammer  -  COHO/NCOHO  Type, 
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ERP,  Pulse  Characteristics.  Target  Number 
Designated  Chaff  Dispenser  -  Type,  Mean  Cross 
Section,  Extent,  Target  Number 

To  properly  output  the  target  data  into  the 
receiver  front  end,  time  and  phase  coherent  with 
all  radar  system  aspects,  requires  a  number 
of  events  to  take  place.  The  target  model  must 
make  the  following  determinations  each  radar 
PR1,  properly  synchronized  with  the  transmitted 
signal  and  the  radar  antenna  pointing  direction, 
considering  both  aspects  of  azimuth  and  eleva¬ 
tion: 

Automatically  Acquire  Mode  and  Pulse  Charac¬ 
teristics 

Obtain  Frequency  Data 

Obtain  Transmitter  Peak  Power  Out  Data 

Pulse  Time  Delay  Relative  to  Inputted  Target 
Range 

Azimuth  Position  Synchronous  with  Antenna 
Azimuth  Angle 

Altitude  Position  Synchronous  with  Antenna 
Elevation  Angle 

Antenna  Platform  Data  Ship  Parameter  Correlated 

Calculate  Doppler  Offset  Frequency  as  a  Func¬ 
tion  of  Slant  Range  Rate 

Calculate  Target  Power  as  a  Function  of  Target 
RCS  and  Radar  Peak  Power 

Calculate  Antenna  Gain  as  a  Function  of  Target 
Position  and  Antenna  Pointing  Angle 

Calculate  one  way  or  two  way  as  a  EUnction  of 
Range. 

Input  Frequency  Loss  Word 

Determine  Chaff  or  Rain  Attenuation  if  applicable. 

Output  Target  Pulse  in  Real  Time  Synchronism 
at  the  Proper  Power  Level  and  Frequency. 

The  above  model  parameters  are  applicable 
to  each  target  inputted  into  a  scenario.  As  the 
number  of  targets  and/or  events  increase,  ad¬ 
ditional  determinations  enter  the  generated  model. 

The  multiple  targets  must  be  properly  range 
ordered  and  asimuth  and  elevation  gated  in  syn¬ 
chronism  with  the  radar.  Each  PRI  all  target  a 
must  be  updated  in  range,  asimuth,  and  eleva¬ 
tion  to  be  properly  outputted  into  the  radar  re¬ 
ceiver. 

Since  these  will  always  be  some  restriction 
on  the  allowed  number  of  rf  channels,  some  de¬ 


gree  of  shadowing  will  probably  be  necessary. 
Path  Loss  Factors 

The  Radar  Range  equation  readily  calculates 
the  signal  strength  a  Radar  receiver  would  see 
given  some  of  the  basic  parameters  of  the  radar's 
operation  (frequency,  power)  and  the  location 
(range)  of  the  target. 

t.4  .  Po  x  G2  x  <r  x  e"2aR 

*  '  (TT'iTjrv^xL- 

The  Radar  Range  equation  in  the  form  shown 
above  makes  a  number  of  assumtions  a)  that 
the  O’  (RCS)  is  fixed,  b)  that  there  is  but  one 
path  between  the  Radar  and  the  target  and  c) 
the  attenuation  and  losses  reamin  invariant. 

In  a  real  life  situation  this  however,  is  not  so 
and  the  simulation  must  take  into  account  these 
variations  to  present  a  realistic  picture. 

The  RCS  variations  have  been  previously 
discussed  together  with  the  multipath  consid¬ 
erations  between  radar  and  target.  Added  to 
the  model  are  two  other  attenuator  factors  in 
addition  to  the  range  loss  considerations,  such 
as;  frequency  losses  and  target  attenuations 
due  to  chaff  or  rain. 

Doppler 

All  moving  targets  generate  a  doppler  off¬ 
set  frequency  relative  to  their  slant  range  rate, 
except  for  those  moving  in  a  concentric  circle 
about  the  radar.  These  targets  are  not  only 
the  maneuverable  targets  inputted  into  the 
simulator,  but  also  the  extended  targets  such 
as  chaff,  rain,  and  sea  clutter. 

The  doppler  effect  causes  the  signal  re¬ 
flected  by  a  moving  target  to  be  offset  in  fre¬ 
quency  by  an  amount  fD  =  2V 

A” 

where  fD  *  Doppler  offset  frequency  (Hz) 

V  =  Relative  velocity  (Slant  Range  Rate)  between 
radar  and  target  (ml an) 

Wavelength  of  currier  frequency  (m) 

In  order  to  properly  output  each  moving 
target  pulse,  phase  coherent  with  the  radar, 
the  proper  doppler  offset  frequency  must  be 
modeled.  This  will  require  two  simulator  cal¬ 
culations.  One,  to  determine  the  instantaneous 
slant  range  rate  as  a  function  of  the  inputted 
velocity  and  heading,  and  second,  calculating 
the  doppler  offset  frequency  or  phase.  These 
calculations  are  obviously  only  performed  on 
those  targets  entering  the  radar  antenna  main 
beam  or  side  lobes. 

An  understanding  of  how  a  radar  processes 
the  incoming  signals  to  differentiate  between 
non-moving  clutter  and  moving  targets  helped 
to  arrive  at  the  solution  implemented. 
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On  a  short  pulse  basis  common  to  most 
radar  system,  it  is  an  impossibility  to  determine 
the  phase  change,  as  a  result  of  doppler  offset.on 
a  single  pulse  basis.  However,  an  examination 
of  successive  PRI' s  shows  that  echos  from  fixed 
targets  remain  constant  throughout,  but  echos 
from  moving  targets  vary  in  amplitude  from 
sweep  to  sweep  at  a  rate  corresponding  to  the 
doppler  frequency. 

Figure  3  illustrates  this  phenomena,  (Taken 
from  Skolnik  "Introduction  to  Radar  System") 


'"'V 


Ffc.  <*--•)  Swxman  on  M  MTI  draft.  r  i«tw  u^Muk  »*  •  ftmcUO"  of 
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By  measuring  or  detecting  this  phase  change 
on  successive  PRI's  a  determination  can  be  made 
of  the  presence  of  moving  targets,  A  radar's 
capability  to  do  this  on  a  2  pulse,  3  pulse,  or  4 
pulse  basis,  gives  an  indication  of  the  degree  of 
MTI  available. 

Two  different  models  can  be  used  in  the 
simulation  of  doppler.  Precision  offset  oscil¬ 
lators  yield  excellent  duplication  of  doppler 
with  but  minor  sidelobes.  Since  however,  one 
oscillator  must  be  dedicated  to  one  speed,  the 
range  of  speed  that  can  be  simulated  are  limited. 
An  alternate  approach  is  the  use  of  phaseshifters, 
with  some  digital  networks  dedicated  to  each 
target  in  view.  Since  the  phase  is  constant  during 
each  transmission  period,  the  sidelobe  level  is 
not  quite  as  good  as  the  individual  oscillator, 
but  this  approach  can  readily  handle  a  multitude 
of  speeds  and  targets.  Typically  a  4  bit  phase- 
shifter  in  an  L  band  system  yields  a  duplicable 
speed  resolution  of  about  +  10  knots  for  a  sim¬ 
ulated  target  speed  from  0  to  +  2,800  knots, 
i.e.  +1/4%. 

The  doppler  for  extended  targets,  which  for 
this  simulation  are  considered  stationary  in 
range  and  azimuth,  is  a  doppler  distribution  of 
the  individual  scatterers  symmetrical  about  zero. 
By  a  correct  choice  of  AM  noise  modulation 
bandwidth,  the  sought  for  doppler  spread  (sym¬ 
metrical  about  zero)  is  duplicated  by  the  AM 
sidebands  of  the  AM  modulating  signals.  The 
rise  and  fall  times  of  the  generated  AM  noise 
signals  previously  discussed,  can  be  controlled 
to  yield  the  spectral  spread  and  thus  the  range 
of  the  simulated  doppler  spread. 


Scan  Modulation  and  Antenna  Gain 

To  continue  on  the  approach  of  creating  as 
realistic  a  model  possible,  actual  antenna  pattern 
data  experimentally  obtained  can  be  utilized  to 
amplitude  modulated  the  simulated  targets.  The 
model  created  considers  the  effect  of  gain,  side 
lobe  structure,  and  half  power  beamwidth.  The 
approach  considered  is  to  store  the  values  of 
antenna  gain  relative  to  peak  in  ROM  with  suf¬ 
ficient  capacity  dependent  upon  the  desired 
resolution  and  system  accuracy.  The  ROM 
address  locations  are  then  determined  by  the 
angle  away  from  boresight.  By  this  technique 
there  is  no  restriction  on  antenna  pattern  sym¬ 
metry.  The  only  restriction  is  based  on  storage 
capacity  and  the  bits  allocated  for  ROM  addres¬ 
sing.  For  the  usual  target  return  loss  the 
antenna  gain  is  used  both  in  the  transmit  link 
and  the  receive  link.  For  the  jammer  case,  it 
is  only  needed  for  the  receive  calculation. 

Extended  Targets  (Chaff,  Rain,  Sea  Clutter) 

Extended  targets  create  an  echo  return 
that  is  no  longer  a  simple  replica  of  the  trans¬ 
mitted  signal;  but  a  superposition  of  many  re¬ 
turns  from  individual  scattering  centers. 

Extended,  or  long  targets  require  somewhat 
different  techniques  to  simulate.  For  one, 
these  targets  do  present  a  shape  of  their  own 
which  has  to  be  superimposed  with  the  radars 
own  parameters. 

In  range,  the  convolution  is  that  of  the  radar 
pulse,  a  square  wave,  with  that  of  the  extended 
target  shape,  such  as  rain  or  chaff.  In  azimuth 
the  Antenna  beam  shape,  as  it  sweeps  across 
the  target,  must  be  imposed  upon  the  azmuthal 
variations  of  the  target. 

To  generate  the  basic  data,  a  convolution 
integral  of  the  type  below  was  used  to  generate 
the  basic  data  known  as  RCS  for  the  short  targets. 

RCS  <9)^=joBJ  ( 0 )  *  ANT  (  T-0)  dT 

For  Chaff,  the  variations  are  two  fold.  The 
vertical  vatiaHons  with  time  can  best  be  described 
as  a  normal  distribution  with  the  upper  2-sigma 
anchored  at  the  level  of  sowing  and  a  mean  fall 
rate  dependent  on  the  material  employed.  Typi¬ 
cally  the  centroid  fall  rate  is  about  250  ft/min- 
ute.  Simultaneously  the  lateral  spread,  per¬ 
pendicular  to  the  direction  of  sow  increases  at 
about  the  same  rate. 

Rain,  though  similar  to  Chaff,  in  certain 
respects  has  some  significant  differences.  For 
one,  once  rain  has  initiated,  with  its  higher  fall 
velocity,  it  can  be  taken  as  (a)  reaching  the 
ground  in  near  zero  time,  and  (b)  that  its  ver¬ 
tical  profile  is  uniform  from  the  ground  up  to 
the  level  of  the  forming  cloud. 
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Also,  while  Chaff  has  some  kind  of  uniform¬ 
ity  along  the  line  of  sow.  Rain  is  a  more  central 
phenomenum.  It  contains  a  near  circular  center 
with  a  density  taper  toward  the  edges.  Again, 
unlike  Chaff,  while  the  reflectivity  decreases 
outward  from  the  center,  it  does  not  taper  to 
zero  but  experiences  a  step  function  near  its 
edges  providing  a  definite  profile. 

Further,  in  the  discussion  of  Doppler,  while 
the  individual  Chaff  particles  have  a  motion  like  a 
falling  leaf,  (i.  e.  significant  oscillatory  motion 
in  the  horizontal  compared  to  the  vertical  drop), 
the  majority  of  the  rain  motion  is  vertical,  and 
any  lateral  motion  due  to  wind  is  uniform  to  all 
rain  drops.  The  last  statement  holds  for  the 
rain  below  the  cloud,  which  is  the  majority  of  the 
rain  field  in  Radar  view,  though  not  within  the 
storm  center  of  the  raincloud.  As  the  latter  sub¬ 
tends  but  a  small  vertical  angle,  it  can  be  ne¬ 
glected  for  search  radar  simulation. 

In  the  discussion  of  the  small  targets,  the 
multipath  problem  was  considered  <i.e.  the 
superposition  of  two  different  signals  (Path  I 
and  Path  II  and  HI,))  In  the  case  of  large  ex¬ 
tended  targets  a  similar  effect  takes  place,  how¬ 
ever,  with  many  many  more  reflectors  delivering 
their  one  reply  at  slightly  different  times  and 
phases. 

In  the  small  targets,  the  effect  was  varia¬ 
tion  of  the  signal,  as  simulated,  of  various  am¬ 
plitudes.  A  finer  resolution  would  not  do  as  the 
Radar  receiver  bandwidth  is  matched  to  the 
transmitted  signal  duration.  For  extended  tar¬ 
gets.  however,  much  longer  than  a  radar  pulse 
width,  the  receiver  can  resolve  such  variations 
though  bandlimited  to  the  signal  bandwidth. 

Thus,  the  random  variations  again  in  the  range 
of  +6  to  -20  dB.  are  superimposed  on  the 
return  signal.  A  noise  pattern  is  used  with  the 
data  varied  in  a  broad  band  manner  up  to  fre¬ 
quencies  compatible  with  the  transmitted  pulse 
length,  i.  e.  Receiver  bandwidth. 

This  high  frequency  noise  (500  KHz  ,for  a 
2  microsecond  Radar)  is  then  superimposed 
in  the  amplitude  patterns  representing  the 
convolution  of  the  target  physical  reflectivity 
profile  and  the  scanning  antenna  pattern. 

For  straight  pulse  Radars,  the  Chaff/Rain 
signal  control  is  complete  at  this  point.  For 
Chirp  systems  one  factor  has  to  be  considered, 
in  that  scatterers  at  different  ranges  reply  with 
different  frequencies  at  the  same  time.  For 
small  targets  a  replication  of  the  FM  pulse  suf¬ 
fices.  For  extended  targets  an  additional  high 
rate  FM  modulation  is  required  to  simulate  the 
receipt  of  multiple  frequencies  at  the  same  time. 

For  the  simulation  of  extended  targets, 
the  modulating  wave  shape  is  modified  to  simu¬ 
late  the  frequency  centroid  as  a  function  of  time 


during  the  return  signals  duration  (Pulse  width, 
plus  target  depth. )  That  new  centroid  wave¬ 
shape  is  further  modulated  with  a  high  frequency 
signal  (higher  than  the  receiver  bandwidth), 
whose  peak  to  peak  excursions  correspond  to 
the  earliest  and  latest  frequency  in  view  at  any 
given  instant. 

The  Sea  return  signal  uses  similar  techr 
niques.  It  differs  from  either  the  Chaff  or  the 
R^in  (longitudical  or  cylindical  symetry)  in  its 
density  pattern. 

While  Rain  or  Chaff  are  isolated  events. 

Sea  Clutter  differs  in  that  (a)  it  is  a  360  degree 
event,  (b)  it  appears  only  at  close  in  ranges, 
less  than  the  15  NM  ships  Radar  horizon  and 
(c)  in  that  the  main  reflecting  elements  are 
not  the  surface  of  the  ocean  and  its  waves,  which 
could  be  subject  to  separate  analytical  treat¬ 
ment,  but  from  all  indications  are  from  the 
spray  patterns  located  above  the  wave. 

The  main  reflecting  spray  patterns  are 
known  to  be  correlated  to  the  wind  direction 
(not  wave  direction)  and  appear  as  slowly 
moving  reflecting  strips  perpendicular  to  the 
wind  direction.  Additionally,  minor  small 
targets  appear  to  fill  in  the  spaces  between  the 
main  scatterers,  though  of  less  scattering 
crossection. 

There  is  one  more  wind  caused  effect,  in 
that  the  mean  reflections  are  maximum  into 
the  wind,  somewhat  less  alee,  with  an  even 
further  reduction  cross  wind  yielding  an  hour¬ 
glass  figure  for  the  clutter  simulation. 

The  complexity  in  forming  a  sea  state  pat¬ 
tern  is  in  the  generation  of  the  cross  wind  lines 
and  minor  scatterers  as  a  function  of  wind  di¬ 
rection;  which  in  general,  correlate  on  a  scan 
to  scan  basis  but  drift  in  general  position  as  a 
function  of  minutes.  The  simulation  of  the 
multiscatter  effects  are  handled  similar  to 
other  extended  targets. 

A  special  problem  to  be  handled  for  Sea 
state  is  its  close  proximity  to  the  Radar.  The 
effect  of  R4  on  extended  targets  is  minor.  E.  g. 
a  3  mile  rain  at  25  miles  has  but  minor  signal 
variations  due  to  range,  effectively  by  the  26 
dB  noise  modulation.  Sea  returns  at  the  range 
from  1  to  15  NM,  however,  have  major  contri¬ 
butions  from  the  R^  law  that  must  be  incorpora¬ 
ted  into  the  model. 

Jammers 

To  complete  the  electromagnetic  environ¬ 
ment,  model,  consideration  must  be  given  to 
expected  threats.  As  a  result  provision  has 
been  made  to  include  simultaneous  coherent  and 
noncoherent  radiation  from  collocated  jammers. 
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The  replication  of  these  jammer  platforms  is 
accomplished  with  independent  ERP  control  and 
various  operation  modes.  Among  these  are 
coherent  range  deceptions  (including  cover 
pulse)  and  inverse  gain,  noncoherent  swept  CW, 
spot  and  barrage  noise,  plus  synchronous,  and 
nonsynchronous  pulses. 


may  be  initiated  at  any  time  by  operator  or 
data  link  intervention. 

MODEL  IMPLEMENTATION 

Hardware  Implementation  /Radar  Interconnect/ 
Control 


Radar  Interface 

In  order  to  achieve  the  degree  of  coherency 
necessary.certain  information  is  required  from 
the  radar.  The  STALO  and  COHO  frequencies 
are  necessary  to  recreate  a  return  echo  with 
the  necessary  degree  of  phase  coherency.  A 
main  bang  trigger  to  synchronize  the  system  and 
antenna  scan  and  platform  information  are  also 
necessary.  Figure  4  indicates  the  major  inter¬ 
laces . 


Figure  4 

Rf  rs  CONCF"! 

Operator  Interface 

To  simplify  use  of  the  system,  a  micro¬ 
processor  controlled  video  display  terminal 
was  selected  out  of  the  many  considered.  Data 
may  be  entered  by  means  of  the  keyboard,  RS- 
232  serial  or  IEEE  STD  475  parallel  data  link 
inputs.  The  operator  provides  the  simulator 
with  information  relating  to  the  aircraft  and  the 
radar.  This  information  includes  the  aircraft 
identification,  signature,  and  motion  dynamics. 
Also  entered  are  the  radar  parameters  includ¬ 
ing  frequency,  amplitude,  and  time  dimensions. 

The  parameters  defining  the  nature  of  the 
chaff  and  the  type  of  jamming  is  entered  for 
each  aircraft  that  carries  these  countermeasure 
capabilities. 

The  data  that  has  been  entered  for  the  air¬ 
craft,  radar,  chaff  and  jammer  is  presented 
on  the  display.  As  the  data  changes,  the  dis¬ 
play  changes.  There  is  a  constant  updating  of 
the  aircraft  position  to  show  its  new  range  and 
bearing  at  any  time  during  the  mission.  Auto¬ 
matic  self  test  and  a  predetermined  scenario 


A  simulator  incorporating  many  of  the 
features  discussed  in  the  previous  section  was 
designed,  fabricated,  tested  and  delivered.  It 
was  designated  as  a  Multiple  Threat  Generator 
(MTG-100)  and  was  integrated  at  the  Signal 
Processing  Laboratory  Radar  at  Rome  Air 
Development  Center  (RADC).  Its  performance 
characteristics  are  included  at  the  end  of  this 
section. 

Figure  5  shows  the  simulator  and  its  radar 
interconnections  and  Figure  6  is  a  simplified 
block  diagram  of  the  simulator  with  its  inter¬ 
connections.  A  more  detailed  block  diagram 
depicting  a  single  channel  of  the  MTG-100  is 
shown  in  Figure  7.  One  such  channel  is  avail¬ 
able  for  each  R.  F.  band  the  simulator  is  to  be 
ope  rated  in. 


I—  COBHM 


tf'Faure  5 


Figure  6 
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Figure  7 


Each  channel  consists  of  two  chains,  a 
coherent  and  a  noncoherent  chain.  The  input 
frequency  of  the  coherent  chain  is  determined 
by  a  signal  derived  from  the  radar  loc  al 
oscillator. 

In  the  system  chosen  as  illustration,  the 
radar  transmitted  frequency  is  80  MHz  above  the 
local  oscillator  input.  The  80  MHz  oscillators 
are  crystal  controlled  VCO's,  one  dedicated  for 
each  target,  allowing  Doppler  shift  simulation 
to  be  achieved  by  offsetting  the  XVCO's. 

The  mixed  product  is  amplified  to  compen¬ 
sate  for  mixer  conversion  loss  and  component 
insertion  losses. 

A  series  of  control  elements  <i.  e. ,  linear 
attenuator,  R.  F.  switch,  digital  attenuator)  are 
used  to  control  the  amplitude  of  the  R.  F.  signal. 
The  30  dB  linear  attenuator  superimposes  the 
noise  modulation  generated  by  digital  ncr  e 
circuits. 

The  R.  F.  switches  are  used  to  pulse  modu¬ 
late  the  signal  (with  pulse  widths  as  narrow  as 
100  nanoseconds  to  full  CW)  with  an  on-off  ratio 
of  120  dB. 

The  actual  control  of  the  R.  F.  circuitry  is 
the  responsibility  of  the  digital  interface  circuitry 
under  the  control  of  a  microprocessor.  The 
microprocessor  performs  the  calculations  for 
the  proper  attenuator  word,  computes  the  range 
delay,  and  sets  the  pulse  width.  In  addition, 
there  is  circuitry  for  azimuth  gating  based  on 
radar  antenna  position  and  beam  width. 

The  digital  attenuator  (.  ldB  steps  to  120  dB) 
is  driven  by  the  CPU  and  digital  interface  cir¬ 
cuitry  to  provide  the  range,  antenna  pattern 
loss  and  other  dynamic  and  static  losses. 

Both  the  coherent  and  noncoherent  signals 
are  combined  at  the  output  and  are  available  at 
a  single  output  port. 


The  R.  F.  coherent  signal  is  combined  with 
a  signal  generated  by  a  voltage  controlled  phase 
lock  loop  oscillator  whose  output  frequency  is 
digitally  selected.  This  second  oscillator  is  the 
R.  F.  source  for  the  noncoherent  chain  which 
provides  simultaneous  jammer  signal  capability. 

The  digital  circuitry.  Figure  8,  consists  of 
three  main  elements:  a  video  terminal,  the 
central  processing  unit,  and  the  interface  hard¬ 
ware  circuitry.  The  video  terminal  is  essen¬ 
tially  the  front  panel  of  the  radar  simulator. 

This  terminal  may  be  used  to  enter  all  data, 
control  commands  and  make  scenario  decisions. 
The  operator  obtains  current  status  and  other 
housekeeping  information  from  the  display. 


Figure  8 


Software  Implementation 

Both  dynamic  real  time  and  static  process¬ 
ing  is  achieved  with  the  assistance  of  two  8080 
microprocessors  with  both  independent  RAM  and 
common  memory.  The  microprocessors  perform 
the  radar  power  and  cross  section  calculations, 
as  well  as  range  ordering  and  bearing  positioning. 
The  computers  also  provide  the  routing  of  data  in 
and  out  of  the  video  terminal  for  both  display 
and  operator  interactivity. 

Updating  of  the  scenario  and  formatting  of 
video  data  is  performed  by  the  CPU's.  The 
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mainline  program  is  stored  in  less  than  8K  of 
PROM.  The  CPU’s  perform  a  range  ordering 
task  and  provide  a  "shadowing"  signal  whenever 
two  targets  are  within  lOps  of  each  other.  The 
range  ordering,  which  at  present  is  a  modified 
"bubble  sort,  "  arranges  the  aircraft  in  the  order 
of  radar  response  at  their  computed  ranges.  This 
order,  which  is  strictly  range  dependent,  is  then 
gated  with  the  antenna  azimuthal  information. 

Each  time  the  azimuth  gate  (representing  the 
antenna  beam  painting  of  a  particular  angular 
sector)  repeats  a  particular  sector,  the  order 
of  the  range  of  targets  could  possibly  be  different. 
A  comparison  is  made  between  each  target,  its 
radial  range  component  is  considered,  and  all 
of  the  targets  are  repositioned. 

The  data  transmitted  to  the  interface  hard¬ 
ware  consists  of  static  data,  computed  once 
each  time  data  is  entered  from  the  terminal, 
and  dynamic  data,  computed  during  the  scenario 
update  about  once  each  5  millisecond  interval. 
Range  and  range  related  phenomena  is  computed 
on  a  50  foot  resolution  interval  and  Doppler 
frequency  from  0  to  ±  2100  knots  in  1  knot  in¬ 
crements. 

In  order  not  to  disturb  the  simulation  when 
new  data  is  available  from  the  CPU's,  two  data 
registers  (Readout  Files)  are  used  in  the  inter¬ 
face  hardware.  Data  is  read  in  and  changed  in 
one,  while  read  out  on  the  other  file.  By  this 
"ping-pong"  approach,  a  smcoth  update  is 
maintained  and  synchronization  between  the  radar 
main  bang  and  the  CPU  clock  is  not  needed. 

Noise  Modeling 

In  order  to  create  as  realistic  a  target 
simulation  as  possible,  the  noise  characteristics 
shown  in  Table  1  were  included  in  the  noise 
generation  circuitry  for  aircraft  and  chaff  targets. 


A)  Each  aircraft's  noise  components  are 
fully  independent  of  other  aircraft. 

B)  The  glint  component  is  chi-square  dis¬ 
tributed,  time  correlated,  target  number 
correlated,  and  frequency  dependent. 

C)  The  fade  component  is  an  empirical  cyclic 
time  only  related  function  which  is  also 
target  correlated. 

For  the  chaff  noise  modulation  imple¬ 
mentation: 

A)  A  Gaussian  distributed,  antenna 
position  related,  random  walk 
pattern  was  used,  plus  - 

B)  A  Gaussian  distributed  time  and 
frequency  dependent  noise  distribution. 

The  noise  seen  by  a  radar  when  receiving 
a  skin  return  is  comprised  of  several  elements 
which  are  both  time  dependent  and  target 
dependent.  One  element  of  noise  is  a  result 
of  temporal  displacement  (GLINT). 

There  is  another  element  that  is  spatial 
contribution  which  is  related  to  antenna 
position,  target  cross  section  and  target 
orientation.  A  further  distinction  has  to  be 
made  between  noise  from  aircraft  returns 
and  that  from  chaff  returns. 

For  a  single  aircraft  the  amplitude  modu¬ 
lation,  whose  -<robability  density  function  is 
a  chi-square  ,  . stribution,  decorrelates  both 
with  time  (pulse  to  pulse)  and  frequency.  It 
also  provides  scan  to  scan  independence.  This 
component  of  noise  represents  the  GLINT 
portion  of  the  noise  modulation. 
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The  second  element  of  the  noise  modu¬ 
lation  is  the  contribution  due  to  target  fading. 
This  fade  pattern  is  a  slow  change  in  target 
return  amplitude  due  to  aspect  angle  changes 
This  spatial  component  was  instrumented 
using  a  cosine4  function  with  a  variation  in 
the  period  as  a  function  of  the  target.  The 
"Fade"  component,  therefore,  was  empirical¬ 
ly  derived  and  assures  different  noise  be¬ 
havior  for  different  targets. 

The  amplitude  range  of  the  total  aircraft 
noise  is  limited  to  30  dB  with  the  mean  at 
6dB.  The  chaff  noise  modulation  has  a  10  dB 
variation  (±5  dB)  with  a  mean  also  of  6  dB. 

This  assures  that  the  mean  return  power  is  the 
same  for  identical  cross  sectional  areas. 


In  the  case  of  chaff,  a  time  invariant  antenna 
position  component  must  be  generated.  This 
models  the  expected  variation  in  noise  as  the 
antenna  sweeps  different  portions  of  the  chaff. 
The  noise  will  be  highly  correlated  for  angular 
changes  which  are  small  compared  to  the  half 
power  beamwidth  of  the  antenna  and  completely 
decorrelated  when  the  antenna  moves  greater 
than  the  half  power  beamwidth. 

Figure  10  is  a  block  diagram  of  the  noise 
generator  for  the  coherent  channel. 


Figure  10 

Four  counter  outputs,  whose  clock  rate  is 
frequency  dependent  based  on  radar  operating 
frequency,  are  added  and  multiplexed  to  pro¬ 
vide  up  to  ten  different  clocking  rates.  The 
multiplexed  outputs  are  used  to  address  the 
"Glint"  ROM.  Each  aircraft  will  then  have  an 
associated  "sample"  rate  based  on  radar  fre¬ 
quency  and  pulse  repetition  rate  and  its  aircraft 
target  number. 

A  separate  slow  clock  is  used  to  provide  the 
seven  least  significant  bits  of  the  "Fade"  ROM 
address.  The  aircraft  number  is  used  to  pro¬ 
vide  the  three  most  significant  address  bits.  In 
this  way,  each  aircraft  has  its  own  unique  "fade" 
pattern. 

The  sum  of  the  two  ROM  outputs  is  then 
selected  by  a  multiplexer  (decision  based  on 
aircraft  or  chaff  event)  and  fed  to  a  digital  to 
analog  (D/A)  converter.  This  analog  voltage  is 
used  to  drive  a  linear  attenuator  which  modulates 
the  R.  F.  signal. 

The  chaff  noise  generation  technique  is 
similar  except  that  the  ROM  addresses  are 
generated  by  both  a  time  frequency  derived 
clock  and  the  antenna  position  obtained  from 
the  synchro  data.  This  combination  models 
the  variation  due  to  chaff  motion  (time)  and 
decorrelation  due  to  frequency.  It  also  accounts 
for  the  non-homogenous  distribution  of  chaff 
particles  for  a  given  chaff  event. 


Noise  Jammer 

While  the  coherent  noise  distribution  is 
determined  by  the  data  stored  in  the  various 
ROM's  dedicated  for  this  task,  it  is  also 
necessary  to  generate  noise  to  be  used  as  mod¬ 
ulation  for  on-board  jammers.  (The  airplane 
is  the  jammer  platform.)  This  is  done  in  an 
analog  circuit  using  a  noise  diode  and  amplifier 
as  source  voltage  for  a  linear  attenuator.  This 
modulator  affects  the  noncoherent  signal  (rep¬ 
resenting  jammers)  only.  For  broad  band 
(barrage)  noise,  it  was  necessary  to  use  a 
somewhat  different  approach  than  amplitude  mod 
ulation.  The  linear  modulator's  modulation 
bandwidth  is  only  about  300  KHz.  Barrage 
noise  should  produce  a  spectral  bandwidth  in 
excess  of  20  MHz.  To  accomplish  this,  the 
same  noise  diode  source  is  used,  amplified, 
and  superimposed  on  the  error  voltage  of  the 
VCO.  The  phase  lock  circuit  cannot  respond 
fast  enough  to  correct  this  noise  voltage  and 
the  result  is  a  noise  modulated  FM  signal.  This 
signal  occupies  a  bandwidth  of  better  than  20 
MHz  and  spectrally  simulates  a  barrage  jam¬ 
mer. 

Antenna  Pattern  Generation 

The  MTG-100  system  is  capable  of  model¬ 
ing  the  effect  of  gain,  side  lobe  structure,  and 
half  power  beamwidth  of  an  actual  or  ideal  an¬ 
tenna  pattern.  The  values  of  antenna  gain  rela¬ 
tive  to  the  peak  are  stored  in  1024  locations 
where  the  address  location  is  determined  by 
the  angle  away  from  boresight.  There  is  no 
restriction  on  the  antenna  pattern  symmetry. 

The  only  restriction  is  based  on  the  storage 
capacity  and  the  bits  allocated  for  ROM  addres¬ 
sing.  Generally,  a  60  to  70  dB  dynamic  range 
may  be  programmed  with  resolution  Bteps  of  .  1 
dB. 

For  the  usual  radar  target  return  loss,  the 
antenna  gain  is  used  both  in  the  transmit  link 
and  the  receive  link.  For  the  jammer  case,  it 
is  only  needed  for  the  receive  calculation. 
Therefore,  only  the  "one  way"  gain  of  the 
antenna  is  stored  in  the  ROM's  and  the  inter¬ 
face  circuitry  doubles  the  gain  in  the  "two  way" 
case. 

If,  as  an  example,  we  store  about  +11°  of 
an  antenna  pattern,  then  we  can  store  values 
every  .  022°  of  the  pattern.  This  should  be  ad¬ 
equate  even  for  the  highest  gain  antenna  in  use 
today, 

Doppler 

Of  the  many  ways  to  implement  a  Doppler 
shift  (i.  e. ,  Serrodyne,  single  sideband  genera¬ 
tor,  etc.),  only  one  method  can  produce  the 
puUe  to  pulse  phase  coherency  needed  along 
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with  low  unwanted  spurious  signals;  that  is, 
using  a  separate  XVCO  per  target.  Refer  to 
Table  2. 

Regardless  of  the  Doppler  processing  tech¬ 
niques  of  the  radar  under  test,  the  Doppler  shift 
associated  with  a  particular  target  must  maintain 
its  offset  frequency  and  phase  continuity.  Be¬ 
cause  the  radar  only  sees  the  target  for  a  small 
portion  of  the  Doppler  cycle,  it  must  take  several 
samples  and  reconstruct  the  Doppler  shift. 
Therefore,  even  though  the  radar  is  not  looking 
at  the  target,  the  Doppler  phase  must  not  con¬ 
tain  discontinuities  or  else  an  error  will  be 
introduced  during  the  next  sample  (pulse)  period. 
In  addition,  unwanted  sidebands  and  carrier 
leak-through,  if  excessive,  can  produce  errors 
during  processing. 

At  present,  the  Serrodyne  method  or  the 
single  sideband  method  at  best  would  produce 
spurious  frequencies  and  carrier  leak-through 
that  would  be  between  20  and  35  dB  below  the 
carrier.  To  assure  that  spurious  outputs  are 
50  to  55  dB  below  the  desired  output,  a  separate 
oscillator  was  used  for  each  target.  Each  of  the 
oscillators  are  continuously  operating  and  are 
assigned  to  individual  targets.  The  outputs  are 
selected  under  CPU  control  when  that  target  is 
to  appear.  The  selector  switch  presents  a  load 
to  the  oscillator  at  all  times,  thereby  eliminating 
possible  pulling  during  switching.  The  oscilla¬ 
tors  (crystal  controlled  VCO's)  are  capable  of 
being  offset  up  to  +  30  KHz  as  a  function  of  a 
DC  control  voltage. 

The  CPU  was  used  to  calculate  the  Doppler 
shift  needed  based  on  operator  inputs  of  velocity 
and  radar  frequency  used.  The  calculated  Dop¬ 
pler  shift  word  was  applied  to  a  digital  to  ana¬ 
log  converter  whose  reference  voltage  provided 
the  scaling  factor  to  convert  the  frequency  word 
to  a  correct  shift  of  the  XVCO  frequency. 

The  simulation  of  chaff  Doppler  presents  a 
different  challenge.  The  chaff  cloud  requires 
a  random  shift  correlated  with  time  and  operat¬ 
ing  frequency.  The  signal  produced  for  ampli¬ 
tude  modulation  of  the  chaff  noise  has  a  compo¬ 
nent  which  is  related  to  both  time  and  frequency. 
By  using  this  component  before  it  is  combined 
with  the  spatial  noise  component,  the  chaff 
Doppler  was  derived  both  as  a  function  of  op¬ 
erating  frequency  and  time.  The  four  least 
significant  bits  of  the  chaff  noise  ROM  were 
used  as  the  control  word  for  the  XVCO  digital 
to  analog  converter.  This  simulates  the  effect 
of  wind  shear,  dipole  tumbling,  turbulence,  and 
falling  of  the  chaff  cloud. 


TABLE  2  DOPPLER  FREQUENCY  SIMUIJlTlON 

REQUIREMENT:  6  TARGETS  FI. US  CHAFF 

TARGET  YEIOCTTY  FROM  -2100  to  *2100  KNOTS 
CHAFF  VELOCITY  FROM  -24  \o  »2I  KNOTS 
SPURIOUS  1*1  .US  FUNDAMENTAL  LEAKAGE  CO  DR  DOWN 
PHASE  COHERENCES  —  EACH  TARGET— 

—  riH.SE  To  PULSE— 

—ANT  JJL-Ul  TO  ANT  beam - 

RADAR  RESOLUTION— -PHASE  DEPENDENT - 8  KNOTS 

APPROACHES  CONSIDERED: 

PHASE  SHIFT— DIGITAL/ ANA1  OG— FUNDAMENT*  *  ./IF 
SINGLE  SIDE  HAND  MODULATION  (SSll) 

MULTIPLE  VOLTAGE  CONTROLI  ED  CRYSTAL 
ObCIU-ATOJtS  (YCXO) 

SELECTED  APPROACH; 

MULTIPLE  VCXO 

ADVANTAGE:  INHERENT  MEMORY 

DLSlltKD  STABILITY 
SPURIOUS  REJECTION 
DESIRED  LINEARITY 

IMPI  EM EN  CATION: 

TARGET  IJl>.OLin  ION:  104  II*  8.2  KNOTS 
CHAFF  RESOLUTION:  ££15  Hz  1.2  KNOTS 


PERFORMANCE  SPECIFICATIONS 


llaiirlinc  _  .OpUt-tis 


Frequency: 

1 2S-0-1 3fi0  MHz 

2000  570U  Mils 
<1  Mils  f-upa) 

0.5  to  1  it  GHz 

Output  Power: 

Culitaiit  Channel:  -C0«»m 
Noncoherent  Channel:  Odllni 

Dynamic  Range: 

>100  till  <0.1  dH  steps) 

Doppler  Simulation: 

0  to  CM  00  knots 

Target  Capability: 

6  AirriMi,  9  Chaff  Events 
(J*©i:»!.  Medium,  J-*rge> 

127  Aircraft  plus 
Chaff  I-Vcnl  r 

Chaff  Motion  A  Effects: 

Stationary 

Air  Muss  Motion, 
Screening.  Altitude 

Environment cl  .*.i  omolirs: 

Clouds,  Snow,  Rain, 
etc. 

Target  RC5 

0. 1  to  1000  meters2 

RCS  Dynamic 

Static  Mean  plus  Scintillation 

Dynamic  Tracking 

Target  Pulse  Width: 

0.3  to  l»0  microseconds 

Target  Range: 

6  to  400  NM 

Target  Atitnuth: 

0  to  3G0° 

Azimuth  Rate: 

4  6  deg roes /second 

Radar /Jammer  Power: 

0  to  100  d»w 

Jammer  Simulation: 

Number 

2 

lloavy  Density  Modi  l 

Type 

Coherent  A  None  olio  rent 

Modes 

Benge  Deception  (including 
Cover  Pulse. )  Inverse  Gain, 
Synchronous  A  Nor.synchro- 
nows  Pulses,  Swept  C\V,  Spot 
Noise,  Rarrage  Noise 

Antenna  Pattern  Data: 

Azimuth  Coverage 

Azimuth  A  Elevation 

Complex  Wava  forms: 

Pocudo  Random 

Rinur.v  Phase  Coding 
100  Megabit  Clock 

Rale 

Identification: 

IFF  SIF.  Mode  4. 

(KI.S) 
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Equipment  Installation 

The  equipment  was  successfully  installed  at 
RADC  with  the  radar  interface  accomplished  via 
a  20  dB  directional  coupler  to  minimize  any 
degradation  of  radar  receiver  sensitivity.  Test 
data  was  obtained  and  the  MTG-100  met  the  per¬ 
formance  characteristics  specified  in  the  con¬ 
tractual  statement  of  work. 

Problems  Encountered 

One  of  the  major  problems  encountered 
during  the  design  phase  of  the  MTG-100  was  in 
the  generation  of  the  doppler  offset  frequencies. 
The  initial  design  concept  basically  ignored  the 
degree  of  phase  coherency  required  to  be  dedi¬ 
cated  to  each  moving  target,  PRI  to  PRI,  and  the 
phase  memory  that  had  to  be  dedicated  to  each 
target  within  the  antenna  beam.  Coupled  with  the 
above  was  the  requirement  that  carrier  leak 
through  be  60  dB  below  the  doppler  offset  fre¬ 
quency.  The  initial  concept  utilized  a  phase 
locked  VCO  switched  in  frequency  to  generate 
each  doppler  offset  frequency  to  be  mixed  with 
the  incoming  radar  L.O.  frequency.  All  the 
requirements  could  be  met  except  for  in  beam 
and  PRI  to  PRI  phase  coherency. 

Two  different  methods  can  be  used  in  the 
simulation  of  doppler  as  has  been  previously 
discussed  in  the  Doppler  model.  Due  to  the  60 
dB  rejection  criteria  imposed  on  the  MTG-100, 
the  dedicated  XVCO  oscillator  approach  was  used. 

RADAR  OPERATOR/MAINTENANCE 
TECHNICIAN  TRAINING 

System  Performance  Requirements  and  Environ¬ 
ment  Control 

If  a  situation  can  exist,  that  requires  judg¬ 
ment  and  operator  skills  to  properly  evaluate  and 
respond  to,  that  situation  needs  to  be  repeatably 
generated  for  proper  training.  If  a  judgment 
may  be  necessary  dependent  upon  various  shadings 
of  displayed  data,  proper  training  can  only  be 
accomplished  by  either  generating  the  actual 
conditions  (flight /jammer  trials)  or  by  having 
available  a  realistic  simulation. 

Both  of  the  above  training  aspects  can  be 
cost  effectively  realized  by  the  use  of  a  properly 
specified  electromagnetic  environment  simulation. 
By  properly  specified,  we  mean  an  environment 
simulation  virtually  undifferentiable  from  the 
real  world.  Add  to  this  a  capability  to  generate 
a  dynamic  multiple -target /threat  environment 
with  chaff  dispensing  aircraft,  sea  clutter, 
and  rain  events,  and  the  ability  to  sum  these  sig¬ 
nals  with  real  world  inputs,  all  sorts  of  training 
scenarios  become  possible. 

Not  only  training  scenarios  are  possible;  in 


addition,  the  simulation  capability  can  be  used 
to  solve  specific  environment/mission  problems. 
Here  are  a  few  examples: 

1.  Place  a  simulated  target  cross  section 
within  an  actual  displayed  chaff  corridor,  vary 
its  RCS  and  determine  the  size  target  that  would 
be  visibly  displayed. 

2.  Do  the  above  varying  the  simulated 
target's  slant  range  rate  to  see  if  a  target’s 
velocity  has  a  bearing  on  detectability . 

3.  Do  "1"  above  to  determine  the  size 
target  that  can  be  seen  through  a  rain  event . 

4.  A  particular  jammer  ERP  level  can  be 
determined  for  both  "burn  through"  and  display 
jamming. 

5.  Data  can  be  extracted  from  unknown 
targets  by  generating  simulated  identical  target 
tracks. 

Simulation  Applicability  to  Training 

The  training  will  only  be  as  good  as  the 
degree  of  realism  available.  If  it  looks  like 
a  simulated  signal,  a  certain  degree  of  training 
can  be  realized,  but  not  the  ultimate.  If  the 
operator  cannot  tell  the  difference,  then  the 
training  is  being  accomplished  under  virtual 
actual  conditions  at  a  fraction  of  the  cost. 

CONCLUSIONS 

To  achieve  a  high  degre  of  operator  com- 
petance  in  interpreting  a  radar  display  in  a 
threat  environment  requires  a  high  degree  of 
signal  realism.  The  target /signal  models  used 
in  the  simulator  discussed  were  based  actually 
upon  data  obtained,  under  a  variety  of  conditions, 
using  various  radar  systems.  Today's  electronic 
technology  allows  the  generation  of  a  simulated 
radar  electromagnetic  environment  virtually 
undifferentiable  from  real  world  returns. 
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